Nov 04 01:31:54.969656 ip-10-0-129-138 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Nov 04 01:31:54.969704 ip-10-0-129-138 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Nov 04 01:31:54.969713 ip-10-0-129-138 systemd[1]: kubelet.service: Failed with result 'resources'.
Nov 04 01:31:54.969973 ip-10-0-129-138 systemd[1]: Failed to start Kubernetes Kubelet.
Nov 04 01:32:05.126604 ip-10-0-129-138 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Nov 04 01:32:05.126618 ip-10-0-129-138 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot ceb81161b9254d65b9e8bc933ac39cd8 --
Nov 04 01:34:01.570827 ip-10-0-129-138 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 01:34:02.008120 ip-10-0-129-138 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:34:02.008120 ip-10-0-129-138 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 01:34:02.008120 ip-10-0-129-138 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:34:02.008120 ip-10-0-129-138 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:34:02.012909 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012782    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:34:02.012909 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012906    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:34:02.012909 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012911    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:34:02.012909 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012914    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:34:02.012909 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012917    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012920    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012924    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012927    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012930    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012932    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012935    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012938    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012940    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012943    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012946    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012948    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012951    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012954    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012956    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012960    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012962    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012965    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012974    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:34:02.013055 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012977    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012980    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012983    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012985    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012988    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012991    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012993    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012996    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.012998    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013001    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013004    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013007    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013009    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013012    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013014    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013018    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013021    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013024    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013027    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013029    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:34:02.013491 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013032    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013034    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013037    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013039    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013042    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013044    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013047    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013049    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013052    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013054    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013056    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013059    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013062    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013064    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013067    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013070    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013072    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013075    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013077    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013079    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:34:02.014757 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013082    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013084    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013086    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013089    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013091    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013095    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013098    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013102    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013104    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013107    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013110    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013112    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013115    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013117    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013120    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013122    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013125    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013128    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013130    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:34:02.015219 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013134    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013138    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013141    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013144    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013147    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013149    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013151    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013154    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013157    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013161    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013165    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013168    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013171    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013173    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013175    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013178    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013180    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013184    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013186    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013189    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:34:02.015718 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013192    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013194    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013197    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.013200    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014033    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014042    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014045    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014048    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014052    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014057    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014060    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014063    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014066    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014068    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014071    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014074    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014076    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014079    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014081    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:34:02.016194 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014084    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014087    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014090    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014092    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014094    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014097    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014099    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014102    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014104    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014106    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014109    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014112    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014115    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014118    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014121    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014124    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014127    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014130    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014133    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014135    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:34:02.016652 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014137    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014140    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014142    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014144    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014147    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014149    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014152    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014154    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014157    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014159    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014161    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014164    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014166    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014169    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014171    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014174    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014177    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014179    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014182    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:34:02.017109 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014184    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014187    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014189    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014192    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014194    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014196    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014199    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014202    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014204    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014207    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014209    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014212    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014215    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014217    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014220    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014222    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014225    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014227    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014230    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014232    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:34:02.017561 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014234    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014237    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014240    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014242    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014245    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014247    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014250    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014252    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014254    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014257    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014259    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014262    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014264    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014267    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014270    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014272    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014276    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014278    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014281    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014285    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:34:02.018065 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014288    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014291    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014294    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014297    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014300    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014302    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014305    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014307    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014310    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014312    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014314    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014318    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014869    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014876    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014879    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014882    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014884    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014887    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014889    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:34:02.018559 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014892    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014895    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014898    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014900    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014903    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014906    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014908    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014911    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014914    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014917    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014919    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014921    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014924    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014926    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014929    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014931    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014934    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014937    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014939    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014942    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:34:02.018978 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014944    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014947    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014949    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014952    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014954    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014960    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014962    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014965    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014967    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014970    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014973    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014975    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014978    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014981    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014983    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014987    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014991    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014994    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.014998    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:34:02.019419 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015001    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015003    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015006    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015008    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015011    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015013    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015016    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015019    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015021    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015024    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015026    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015030    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015033    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015037    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015040    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015042    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015045    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015047    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015051    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015053    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:34:02.019877 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015056    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015058    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015060    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015063    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015066    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015068    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015071    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015073    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015076    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015078    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015081    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015083    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015086    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015088    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015091    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015094    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015096    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015098    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015100    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:34:02.020470 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015103    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015105    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015108    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015110    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015113    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015117    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015121    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015124    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015127    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015129    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015131    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015135    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015138    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015140    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015142    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015145    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015147    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015150    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015153    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:34:02.020902 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015155    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.015158    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016099    2575 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016112    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016119    2575 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016124    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016128    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016132    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016136    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016140    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016143    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016146    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016150    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016154    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016157    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016160    2575 flags.go:64] FLAG: --cgroup-root=""
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016163    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016166    2575 flags.go:64] FLAG: --client-ca-file=""
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016169    2575 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016176    2575 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016181    2575 flags.go:64] FLAG: --cluster-domain=""
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016185    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016188    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016192    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 01:34:02.021331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016196    2575 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016201    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016204    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016208    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016211    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016215    2575 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016218    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016221    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016224    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016227    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016231    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016235    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016238    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016241    2575 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016244    2575 flags.go:64] FLAG: --enable-server="true"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016247    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016251    2575 flags.go:64] FLAG: --event-burst="100"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016254    2575 flags.go:64] FLAG: --event-qps="50"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016258    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016262    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016267    2575 flags.go:64] FLAG: --eviction-hard=""
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016274    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016281    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016285    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016289    2575 flags.go:64] FLAG: --eviction-soft=""
Nov 04 01:34:02.021888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016292    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016295    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016300    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016303    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016306    2575 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016309    2575 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016312    2575 flags.go:64] FLAG: --feature-gates=""
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016317    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016320    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016324    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016327    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016330    2575 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016333    2575 flags.go:64] FLAG: --help="false"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016336    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016340    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016343    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016345    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016351    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016356    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016377    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016380    2575 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016383    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016386    2575 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 01:34:02.022560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016389    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016392    2575 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016395    2575 flags.go:64] FLAG: --kube-reserved=""
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016398    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016401    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016404    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016407    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016410    2575 flags.go:64] FLAG: --lock-file=""
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016413    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016416    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016420    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016427    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016430    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016434    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016438    2575 flags.go:64] FLAG: --logging-format="text"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016457    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016463    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016468    2575 flags.go:64] FLAG: --manifest-url=""
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016472    2575 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016477    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016481    2575 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016485    2575 flags.go:64] FLAG: --max-pods="110"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016487    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016490    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016493    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 01:34:02.023093 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016496    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016499    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016502    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016505    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016513    2575 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016516    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016519    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016523    2575 flags.go:64] FLAG: --pod-cidr=""
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016526    2575 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016528    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016531    2575 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016535    2575 flags.go:64] FLAG: --port="10250"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016540    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016545    2575 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0856cc5f13ab9b33f"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016550    2575 flags.go:64] FLAG: --qos-reserved=""
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016553    2575 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016557    2575 flags.go:64] FLAG: --register-node="true"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016560    2575 flags.go:64] FLAG: --register-with-taints=""
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016566    2575 flags.go:64] FLAG: --registry-burst="10"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016569    2575 flags.go:64] FLAG: --registry-qps="5"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016572    2575 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016575    2575 flags.go:64] FLAG: --reserved-memory=""
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016580    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016583    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016586    2575 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016590    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 01:34:02.023672 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016592    2575 flags.go:64] FLAG: --runonce="false"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016595    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016599    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016601    2575 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016604    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016607    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016610    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016614    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016619    2575 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016624    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016629    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016633    2575 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016636    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016639    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016642    2575 flags.go:64] FLAG: --system-cgroups=""
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016645    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016651    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016654    2575 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016657    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016662    2575 flags.go:64] FLAG: --tls-min-version=""
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016665    2575 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016668    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016671    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016674    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016678    2575 flags.go:64] FLAG: --v="2"
Nov 04 01:34:02.024286 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016682    2575 flags.go:64] FLAG: --version="false"
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016687    2575 flags.go:64] FLAG: --vmodule=""
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016691    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.016699    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016867    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016872    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016875    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016878    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016881    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016885    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016889    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016891    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016895    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016898    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016900    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016902    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016905    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016907    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016909    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:34:02.024879 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016912    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016916    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016919    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016921    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016924    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016927    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016929    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016932    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016936    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016941    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016945    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016949    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016951    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016954    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016957    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016959    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016963    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016969    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016971    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:34:02.025321 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016974    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016977    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016979    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016982    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016984    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016986    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016989    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016992    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016994    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016997    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.016999    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017002    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017004    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017006    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017009    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017011    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017016    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017020    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017025    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:34:02.025775 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017028    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017031    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017033    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017036    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017038    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017040    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017043    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017046    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017048    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017051    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017055    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017058    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017061    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017064    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017066    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017068    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017071    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017073    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017075    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017078    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:34:02.026215 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017080    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017083    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017085    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017087    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017090    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017093    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017097    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017102    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017106    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017110    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017113    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017116    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017119    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017122    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017124    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017127    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017129    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017131    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017134    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017136    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:34:02.026749 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017139    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:34:02.027201 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017142    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:34:02.027201 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017146    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:34:02.027201 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017148    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:34:02.027201 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017150    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:34:02.027201 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017153    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:34:02.027201 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017156    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:34:02.027201 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017159    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:34:02.027201 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017161    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:34:02.027201 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017164    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:34:02.027201 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017167    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:34:02.027201 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017169    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:34:02.027201 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017172    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:34:02.027201 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.017209    2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:34:02.027201 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017248    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:34:02.027201 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017256    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:34:02.027201 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017262    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017268    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017271    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017274    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017277    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017280    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017282    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017285    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017287    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017290    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017292    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017294    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017297    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017299    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017302    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017306    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017309    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017311    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017315    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:34:02.027579 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017317    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017320    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017322    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017328    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017332    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017336    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017340    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017345    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017348    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017350    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017353    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017355    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017358    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017360    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017363    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017365    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017367    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017370    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017372    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:34:02.028030 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017375    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017377    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017380    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017382    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017384    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017387    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017389    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017392    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017394    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017398    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017400    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017403    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017406    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017410    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017414    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017419    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017423    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017428    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017431    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017433    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:34:02.028478 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017436    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017438    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017440    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017456    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017459    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017462    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017465    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017467    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017469    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017472    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017475    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017477    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017480    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017482    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017484    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017487    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017491    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017502    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017507    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017509    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:34:02.028931 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017512    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017515    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017518    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017521    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017524    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017527    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017530    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017532    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017535    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017537    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017541    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017543    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017546    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017548    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017552    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017556    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017559    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017562    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017565    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:34:02.029372 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017568    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017572    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017577    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017582    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017586    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017589    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.017593    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.025123    2575 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.025144    2575 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.025161    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.025167    2575 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025230    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025236    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025239    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025242    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025245    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025248    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025250    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025253    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:34:02.029808 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025256    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025258    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025261    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025263    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025267    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025270    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025272    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025275    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025277    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025280    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025282    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025284    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025287    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025289    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025292    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025295    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025297    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025300    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025302    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:34:02.030250 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025304    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025307    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025311    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025314    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025316    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025319    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025321    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025324    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025328    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025333    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025337    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025340    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025343    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025345    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025348    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025351    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025353    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025356    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025359    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:34:02.030819 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025362    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025364    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025366    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025369    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025372    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025374    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025376    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025379    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025381    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025384    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025386    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025389    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025391    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025393    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025396    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025400    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025403    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025405    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025407    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025410    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:34:02.031248 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025412    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025415    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025418    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025420    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025422    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025425    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025427    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025430    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025432    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025435    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025437    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025439    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025442    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025465    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025469    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025471    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025474    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025477    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025479    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025481    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:34:02.031726 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025484    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025486    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025489    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025491    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025494    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025497    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025501    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025505    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025509    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025511    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025514    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025516    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025519    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025521    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025524    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025527    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025529    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025531    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025534    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:34:02.032346 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025536    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:34:02.033051 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.025569    2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:34:02.033051 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025714    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:34:02.033051 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025719    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:34:02.033051 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025722    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:34:02.033051 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025725    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:34:02.033051 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025727    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:34:02.033051 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025730    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:34:02.033051 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025732    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:34:02.033051 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025735    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:34:02.033051 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025737    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:34:02.033051 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025740    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:34:02.033051 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025743    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:34:02.033051 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025745    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:34:02.033051 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025747    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:34:02.033051 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025750    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:34:02.033051 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025752    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025755    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025758    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025760    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025764    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025767    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025769    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025771    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025774    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025777    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025780    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025782    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025785    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025787    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025790    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025792    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025795    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025797    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025800    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025802    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:34:02.033549 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025805    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025808    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025810    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025813    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025816    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025819    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025821    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025824    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025826    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025829    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025833    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025837    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025840    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025843    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025845    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025848    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025851    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025853    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025856    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:34:02.034005 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025858    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025861    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025863    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025866    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025868    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025870    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025873    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025875    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025877    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025880    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025882    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025885    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025887    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025890    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025892    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025899    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025902    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025905    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025907    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025910    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:34:02.034500 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025912    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025915    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025917    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025919    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025921    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025924    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025926    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025929    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025931    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025934    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025936    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025939    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025942    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025944    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025947    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025949    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025952    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025954    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025957    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025959    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:34:02.034991 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025961    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:34:02.035490 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025964    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:34:02.035490 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025966    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:34:02.035490 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025969    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:34:02.035490 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025971    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:34:02.035490 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025974    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:34:02.035490 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025976    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:34:02.035490 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025978    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:34:02.035490 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025982    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:34:02.035490 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025986    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:34:02.035490 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025989    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:34:02.035490 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025992    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:34:02.035490 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.025994    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:34:02.035490 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.026018    2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:34:02.035490 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026050    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:34:02.035490 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026054    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026056    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026059    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026061    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026064    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026067    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026069    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026071    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026074    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026077    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026079    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026082    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026084    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026087    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026090    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026092    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026097    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026100    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026102    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:34:02.035850 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026104    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026107    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026109    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026112    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026115    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026117    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026121    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026125    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026127    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026130    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026133    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026135    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026138    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026140    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026142    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026145    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026147    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026150    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026152    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:34:02.036284 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026155    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026157    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026159    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026162    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026164    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026166    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026169    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026171    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026174    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026176    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026180    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026183    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026185    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026188    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026190    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026193    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026195    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026198    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026200    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026203    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:34:02.036719 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026205    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026207    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026210    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026213    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026215    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026217    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026220    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026223    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026225    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026228    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026231    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026233    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026236    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026238    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026240    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026242    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026247    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026250    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026252    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026255    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:34:02.037172 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026257    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026260    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026264    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026267    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026269    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026271    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026274    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026276    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026279    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026281    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026284    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026286    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026289    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026291    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026294    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026296    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026299    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026302    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026304    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026307    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:34:02.037642 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026309    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:34:02.038086 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026311    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:34:02.038086 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026315    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:34:02.038086 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026317    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:34:02.038086 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026319    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:34:02.038086 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026322    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:34:02.038086 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:02.026324    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:34:02.038086 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.026382    2575 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 01:34:02.038086 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.028657    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 04 01:34:02.038086 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.029546    2575 server.go:1014] "Starting client certificate rotation"
Nov 04 01:34:02.038086 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.029646    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 01:34:02.038086 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.030377    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 01:34:02.038086 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.031512    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:34:02.038086 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.034332    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:34:02.047528 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.047498    2575 log.go:25] "Validated CRI v1 runtime API"
Nov 04 01:34:02.050976 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.050953    2575 log.go:25] "Validated CRI v1 image API"
Nov 04 01:34:02.051839 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.051813    2575 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 01:34:02.054540 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.054516    2575 fs.go:135] Filesystem UUIDs: map[308bd465-3a5b-422e-b29a-7f0c430bde57:/dev/nvme0n1p3 666f8308-26f6-4156-8c6b-aab85c4569c0:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Nov 04 01:34:02.054590 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.054540    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 04 01:34:02.059026 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.058892    2575 manager.go:217] Machine: {Timestamp:2026-11-04 01:34:02.058367824 +0000 UTC m=+0.352975383 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3211398 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2463b48f9f5bacb0d3436e9d663aae SystemUUID:ec2463b4-8f9f-5bac-b0d3-436e9d663aae BootID:ceb81161-b925-4d65-b9e8-bc933ac39cd8 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:72:78:8a:cd:b3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:72:78:8a:cd:b3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:56:b2:6c:cd:bf:d4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 01:34:02.059026 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.059021    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 01:34:02.059143 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.059110    2575 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 01:34:02.062091 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.062052    2575 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 01:34:02.062237 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.062093    2575 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-138.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 01:34:02.062277 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.062240    2575 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 01:34:02.062277 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.062249    2575 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 01:34:02.062277 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.062272    2575 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:34:02.062382 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.062290    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:34:02.063191 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.063179    2575 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 01:34:02.066962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.066944    2575 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 01:34:02.067121 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.067112    2575 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 01:34:02.067846 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.067833    2575 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 01:34:02.067884 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.067851    2575 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 01:34:02.067884 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.067867    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 01:34:02.067884 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.067878    2575 kubelet.go:405] "Adding apiserver pod source"
Nov 04 01:34:02.067979 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.067886    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 01:34:02.068252 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.068240    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:34:02.070620 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.070599    2575 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 01:34:02.075461 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.075429    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:34:02.076196 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.076183    2575 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 01:34:02.076243 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.076214    2575 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 01:34:02.076393 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.076382    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 01:34:02.076435 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.076399    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 01:34:02.076435 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.076406    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 01:34:02.076435 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.076411    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 01:34:02.076435 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.076417    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 01:34:02.076435 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.076423    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 01:34:02.076948 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.076938    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 01:34:02.076948 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.076948    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 01:34:02.077000 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.076955    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 01:34:02.077000 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.076962    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 01:34:02.077000 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.076972    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 01:34:02.077000 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.076981    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 01:34:02.077665 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.077645    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 01:34:02.077665 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.077656    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 01:34:02.082335 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.082318    2575 server.go:1263] "Started kubelet"
Nov 04 01:34:02.082477 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.082418    2575 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 01:34:02.085819 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.082460    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 01:34:02.086118 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.086035    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 01:34:02.086717 ip-10-0-129-138 systemd[1]: Started Kubernetes Kubelet.
Nov 04 01:34:02.088186 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.087380    2575 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 01:34:02.088815 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.088728    2575 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 01:34:02.091889 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:02.091869    2575 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 04 01:34:02.093653 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.093627    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 01:34:02.094570 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.094552    2575 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 01:34:02.094698 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:02.094675    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-138.us-west-1.compute.internal\" not found"
Nov 04 01:34:02.095174 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.095124    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 01:34:02.095174 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.095136    2575 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 01:34:02.095174 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.095155    2575 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 01:34:02.095297 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.095267    2575 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 01:34:02.095297 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.095275    2575 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 01:34:02.095352 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.095313    2575 factory.go:55] Registering systemd factory
Nov 04 01:34:02.095352 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.095327    2575 factory.go:223] Registration of the systemd container factory successfully
Nov 04 01:34:02.095582 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.095566    2575 factory.go:153] Registering CRI-O factory
Nov 04 01:34:02.095582 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.095581    2575 factory.go:223] Registration of the crio container factory successfully
Nov 04 01:34:02.095692 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.095622    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 01:34:02.095692 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.095636    2575 factory.go:103] Registering Raw factory
Nov 04 01:34:02.095692 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.095647    2575 manager.go:1199] Started watching for new ooms in manager
Nov 04 01:34:02.096538 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.096525    2575 manager.go:319] Starting recovery of all containers
Nov 04 01:34:02.106817 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.106790    2575 manager.go:324] Recovery completed
Nov 04 01:34:02.108524 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:02.108386    2575 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Nov 04 01:34:02.111369 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.111350    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:34:02.113254 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.113233    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:34:02.113318 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.113271    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:34:02.113318 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.113282    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:34:02.113907 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.113894    2575 cpu_manager.go:226] "Starting" policy="none"
Nov 04 01:34:02.113907 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.113905    2575 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 01:34:02.114001 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.113919    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:34:02.114001 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.113924    2575 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 01:34:02.114686 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.114669    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:34:02.114756 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.114700    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:34:02.114756 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.114710    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:34:02.116191 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.116179    2575 policy_none.go:50] "Start"
Nov 04 01:34:02.116228 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.116195    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 01:34:02.116228 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.116209    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 01:34:02.118193 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.118181    2575 policy_none.go:44] "Start"
Nov 04 01:34:02.164413 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.164229    2575 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 01:34:02.168724 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.168695    2575 manager.go:341] "Starting Device Plugin manager"
Nov 04 01:34:02.168863 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:02.168739    2575 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 01:34:02.168863 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.168750    2575 server.go:87] "Starting device plugin registration server"
Nov 04 01:34:02.169057 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.169043    2575 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 01:34:02.169115 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.169058    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 01:34:02.169290 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.169264    2575 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 01:34:02.169398 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.169358    2575 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 01:34:02.169398 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.169367    2575 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 01:34:02.169865 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:02.169841    2575 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 01:34:02.169930 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:02.169905    2575 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-138.us-west-1.compute.internal\" not found"
Nov 04 01:34:02.226762 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.226724    2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-138.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 04 01:34:02.226762 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:02.226736    2575 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-138.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 04 01:34:02.228865 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:02.226734    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-138.us-west-1.compute.internal.18e4b39dd6065ad3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-138.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-138.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 01:34:02.082278099 +0000 UTC m=+0.376885652,LastTimestamp:2026-11-04 01:34:02.082278099 +0000 UTC m=+0.376885652,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-138.us-west-1.compute.internal,}"
Nov 04 01:34:02.248522 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.248484    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 01:34:02.250082 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.250055    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 01:34:02.250082 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.250081    2575 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 01:34:02.250281 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.250107    2575 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 01:34:02.250281 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:02.250142    2575 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 04 01:34:02.269584 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.269492    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:34:02.270625 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.270605    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:34:02.270746 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.270638    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:34:02.270746 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.270652    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:34:02.270746 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.270679    2575 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.313580 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.313540    2575 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 01:34:02.313756 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.313623    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gk9wc"
Nov 04 01:34:02.350601 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.350556    2575 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal"]
Nov 04 01:34:02.350733 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.350638    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:34:02.352262 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.352244    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:34:02.352366 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.352275    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:34:02.352366 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.352285    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:34:02.354639 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.354621    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:34:02.354806 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.354787    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.354874 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.354831    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:34:02.355640 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.355625    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:34:02.355715 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.355658    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:34:02.355715 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.355667    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:34:02.355715 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.355627    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:34:02.355823 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.355743    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:34:02.355823 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.355767    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:34:02.357923 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.357904    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.357992 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.357934    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:34:02.359722 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.359703    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:34:02.359833 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.359739    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:34:02.359833 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.359752    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:34:02.386610 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.386581    2575 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:34:02.388040 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.388021    2575 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:34:02.390570 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:02.390552    2575 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-138.us-west-1.compute.internal\" not found" node="ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.393724 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:02.393698    2575 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-138.us-west-1.compute.internal\" not found" node="ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.396712 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.396688    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c58b01dca1916bc539c28fce5659b181-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal\" (UID: \"c58b01dca1916bc539c28fce5659b181\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.396819 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.396717    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c58b01dca1916bc539c28fce5659b181-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal\" (UID: \"c58b01dca1916bc539c28fce5659b181\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.396819 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.396739    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5d9a5980aefdf4e226d40970221f0eb6-config\") pod \"kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal\" (UID: \"5d9a5980aefdf4e226d40970221f0eb6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.486016 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.485977    2575 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:34:02.487670 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.487650    2575 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:34:02.488246 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.488223    2575 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.488285 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:02.488259    2575 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-138.us-west-1.compute.internal\": node \"ip-10-0-129-138.us-west-1.compute.internal\" not found"
Nov 04 01:34:02.496072 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.496043    2575 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.497184 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.497167    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c58b01dca1916bc539c28fce5659b181-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal\" (UID: \"c58b01dca1916bc539c28fce5659b181\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.497240 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.497201    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c58b01dca1916bc539c28fce5659b181-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal\" (UID: \"c58b01dca1916bc539c28fce5659b181\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.497240 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.497216    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5d9a5980aefdf4e226d40970221f0eb6-config\") pod \"kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal\" (UID: \"5d9a5980aefdf4e226d40970221f0eb6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.497300 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.497242    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c58b01dca1916bc539c28fce5659b181-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal\" (UID: \"c58b01dca1916bc539c28fce5659b181\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.497300 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.497253    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5d9a5980aefdf4e226d40970221f0eb6-config\") pod \"kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal\" (UID: \"5d9a5980aefdf4e226d40970221f0eb6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.497300 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.497274    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c58b01dca1916bc539c28fce5659b181-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal\" (UID: \"c58b01dca1916bc539c28fce5659b181\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.557972 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.557878    2575 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:34:02.558142 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.558019    2575 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.615659 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.615620    2575 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:34:02.694008 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.693966    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:02.697236 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:02.697210    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:34:03.029603 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.029563    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 04 01:34:03.030146 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.029796    2575 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 01:34:03.030146 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.029813    2575 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 01:34:03.030146 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.029800    2575 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 01:34:03.030146 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:03.029844    2575 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://aa64b852660404cf9a45925727badae8-e04a1860fe467e96.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-129-138.us-west-1.compute.internal.18e4b39dd7df7f91\": read tcp 10.0.129.138:58772->3.151.182.137:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-129-138.us-west-1.compute.internal.18e4b39dd7df7f91  default   5253 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-138.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-138.us-west-1.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-138.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 01:34:02 +0000 UTC,LastTimestamp:2026-11-04 01:34:02.270656033 +0000 UTC m=+0.565263587,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-138.us-west-1.compute.internal,}"
Nov 04 01:34:03.068742 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.068709    2575 apiserver.go:52] "Watching apiserver"
Nov 04 01:34:03.094536 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.094498    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 04 01:34:03.319040 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.318943    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 01:29:02 +0000 UTC" deadline="2028-07-13 15:18:31.77697576 +0000 UTC"
Nov 04 01:34:03.319040 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.318980    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14821h44m28.457998406s"
Nov 04 01:34:03.347123 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.347086    2575 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 01:34:03.347403 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.347386    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-z992q","openshift-multus/multus-additional-cni-plugins-hh4tb","openshift-ovn-kubernetes/ovnkube-node-6lfbv","openshift-dns/node-resolver-zpmgq","openshift-multus/multus-hcvld","openshift-multus/network-metrics-daemon-q7q6c","openshift-network-operator/iptables-alerter-2sbtc","kube-system/konnectivity-agent-kwbdd","kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx","openshift-cluster-node-tuning-operator/tuned-hhttg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal","openshift-network-diagnostics/network-check-target-hbs2c","kube-system/global-pull-secret-syncer-hjqq9"]
Nov 04 01:34:03.349884 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.349867    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2sbtc"
Nov 04 01:34:03.351838 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.351818    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:03.351944 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:03.351921    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:03.353899 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.353880    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.355857 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.355838    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zpmgq"
Nov 04 01:34:03.357978 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.357958    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.360036 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.360019    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:03.360102 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:03.360077    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:03.362087 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.362071    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:03.362161 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:03.362145    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:03.364203 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.364184    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z992q"
Nov 04 01:34:03.366573 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.366547    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.369094 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.369071    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.371322 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.371302    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.373628 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.373602    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kwbdd"
Nov 04 01:34:03.396654 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.396626    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 01:34:03.402024 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402003    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7647t\" (UniqueName: \"kubernetes.io/projected/a8f004fb-8c9c-4efb-b60d-358e32ba5120-kube-api-access-7647t\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.402153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402037    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-system-cni-dir\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.402153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402107    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-etc-kubernetes\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.402254 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402180    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-dbus\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:03.402254 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402207    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aa398d49-e1ee-41df-8785-b765e1f35b18-hosts-file\") pod \"node-resolver-zpmgq\" (UID: \"aa398d49-e1ee-41df-8785-b765e1f35b18\") " pod="openshift-dns/node-resolver-zpmgq"
Nov 04 01:34:03.402254 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402231    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a8f004fb-8c9c-4efb-b60d-358e32ba5120-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.402377 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402259    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:03.402377 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402304    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-run-netns\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.402377 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402353    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-cni-bin\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.402614 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402393    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6fjs\" (UniqueName: \"kubernetes.io/projected/52cfdb11-898a-4673-97c5-ecddc1385a03-kube-api-access-m6fjs\") pod \"iptables-alerter-2sbtc\" (UID: \"52cfdb11-898a-4673-97c5-ecddc1385a03\") " pod="openshift-network-operator/iptables-alerter-2sbtc"
Nov 04 01:34:03.402614 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402438    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-tuned\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.402614 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402566    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37d34756-13c3-4993-a3d1-07f68eada6b0-tmp\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.402614 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402598    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvlmf\" (UniqueName: \"kubernetes.io/projected/37d34756-13c3-4993-a3d1-07f68eada6b0-kube-api-access-xvlmf\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.402757 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402621    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-slash\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.402757 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402679    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzftg\" (UniqueName: \"kubernetes.io/projected/aa398d49-e1ee-41df-8785-b765e1f35b18-kube-api-access-lzftg\") pod \"node-resolver-zpmgq\" (UID: \"aa398d49-e1ee-41df-8785-b765e1f35b18\") " pod="openshift-dns/node-resolver-zpmgq"
Nov 04 01:34:03.402757 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402708    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58s5t\" (UniqueName: \"kubernetes.io/projected/20296e8e-e25f-40e1-9b2b-4971b7e69141-kube-api-access-58s5t\") pod \"node-ca-z992q\" (UID: \"20296e8e-e25f-40e1-9b2b-4971b7e69141\") " pod="openshift-image-registry/node-ca-z992q"
Nov 04 01:34:03.402757 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402729    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-var-lib-openvswitch\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.402925 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402764    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-run-k8s-cni-cncf-io\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.402925 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402793    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-var-lib-kubelet\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.402925 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402819    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-registration-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.402925 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402861    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-node-log\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.402925 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402899    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52cfdb11-898a-4673-97c5-ecddc1385a03-host-slash\") pod \"iptables-alerter-2sbtc\" (UID: \"52cfdb11-898a-4673-97c5-ecddc1385a03\") " pod="openshift-network-operator/iptables-alerter-2sbtc"
Nov 04 01:34:03.403146 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402929    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-modprobe-d\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.403146 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.402981    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/480e250d-44a3-4d5c-987d-22737107544e-agent-certs\") pod \"konnectivity-agent-kwbdd\" (UID: \"480e250d-44a3-4d5c-987d-22737107544e\") " pod="kube-system/konnectivity-agent-kwbdd"
Nov 04 01:34:03.403146 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403038    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/192e2fec-fefb-4bc5-9516-ed7464963a10-ovnkube-script-lib\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.403146 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403086    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8f004fb-8c9c-4efb-b60d-358e32ba5120-cni-binary-copy\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.403146 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403111    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxtjk\" (UniqueName: \"kubernetes.io/projected/67ed6df2-450d-4705-8a11-f98d5b507dc4-kube-api-access-gxtjk\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.403146 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403132    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-lib-modules\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.403382 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403156    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a8f004fb-8c9c-4efb-b60d-358e32ba5120-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.403382 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403206    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-run-netns\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.403382 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403243    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vz9t\" (UniqueName: \"kubernetes.io/projected/7acf7f6e-efb8-498e-8066-a60fbedecd49-kube-api-access-4vz9t\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.403382 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403266    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-cni-netd\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.403382 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403286    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs\") pod \"network-metrics-daemon-q7q6c\" (UID: \"e62bee01-e717-4484-bb3d-ff4af522a355\") " pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:03.403382 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403304    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/52cfdb11-898a-4673-97c5-ecddc1385a03-iptables-alerter-script\") pod \"iptables-alerter-2sbtc\" (UID: \"52cfdb11-898a-4673-97c5-ecddc1385a03\") " pod="openshift-network-operator/iptables-alerter-2sbtc"
Nov 04 01:34:03.403382 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403347    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-var-lib-cni-bin\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.403707 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403390    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-sysctl-conf\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.403707 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403418    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-systemd\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.403707 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403498    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-var-lib-kubelet\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.403707 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403530    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/480e250d-44a3-4d5c-987d-22737107544e-konnectivity-ca\") pod \"konnectivity-agent-kwbdd\" (UID: \"480e250d-44a3-4d5c-987d-22737107544e\") " pod="kube-system/konnectivity-agent-kwbdd"
Nov 04 01:34:03.403707 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403556    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 01:34:03.403707 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403558    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/192e2fec-fefb-4bc5-9516-ed7464963a10-ovnkube-config\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.403707 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403591    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxtbz\" (UniqueName: \"kubernetes.io/projected/192e2fec-fefb-4bc5-9516-ed7464963a10-kube-api-access-gxtbz\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.403707 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403609    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa398d49-e1ee-41df-8785-b765e1f35b18-tmp-dir\") pod \"node-resolver-zpmgq\" (UID: \"aa398d49-e1ee-41df-8785-b765e1f35b18\") " pod="openshift-dns/node-resolver-zpmgq"
Nov 04 01:34:03.403707 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403645    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8f004fb-8c9c-4efb-b60d-358e32ba5120-cnibin\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.403707 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403684    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-cnibin\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.404161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403724    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-kubelet-config\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:03.404161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403746    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-sysctl-d\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.404161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403768    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-etc-openvswitch\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.404161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403783    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-cni-dir\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.404161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403797    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-hostroot\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.404161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403820    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-conf-dir\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.404161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403856    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-run-multus-certs\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.404161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403882    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20296e8e-e25f-40e1-9b2b-4971b7e69141-host\") pod \"node-ca-z992q\" (UID: \"20296e8e-e25f-40e1-9b2b-4971b7e69141\") " pod="openshift-image-registry/node-ca-z992q"
Nov 04 01:34:03.404161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403927    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-sys-fs\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.404161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403945    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-sysconfig\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.404161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.403974    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-kubelet\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.404161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404005    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-socket-dir-parent\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.404161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404025    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/20296e8e-e25f-40e1-9b2b-4971b7e69141-serviceca\") pod \"node-ca-z992q\" (UID: \"20296e8e-e25f-40e1-9b2b-4971b7e69141\") " pod="openshift-image-registry/node-ca-z992q"
Nov 04 01:34:03.404161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404044    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-device-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.404161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404067    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-run\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.404161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404091    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-host\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.404161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404092    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:34:03.404161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404104    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 01:34:03.405021 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404115    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/192e2fec-fefb-4bc5-9516-ed7464963a10-env-overrides\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.405021 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404104    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lhd4j\""
Nov 04 01:34:03.405021 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404145    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8f004fb-8c9c-4efb-b60d-358e32ba5120-system-cni-dir\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.405021 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404174    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-var-lib-cni-multus\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.405021 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404193    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-sys\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.405021 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404217    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-run-systemd\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.405021 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404238    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-log-socket\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.405021 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404261    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/192e2fec-fefb-4bc5-9516-ed7464963a10-ovn-node-metrics-cert\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.405021 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404284    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/67ed6df2-450d-4705-8a11-f98d5b507dc4-cni-binary-copy\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.405021 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404307    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.405021 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404355    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-etc-selinux\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.405021 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404395    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c8rj\" (UniqueName: \"kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj\") pod \"network-check-target-hbs2c\" (UID: \"ca08235e-943c-4d8d-939d-4ceaedd9cf92\") " pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:03.405021 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404431    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-systemd-units\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.405021 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404478    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-run-openvswitch\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.405021 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404514    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-run-ovn-kubernetes\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.405021 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404541    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.405021 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404559    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8f004fb-8c9c-4efb-b60d-358e32ba5120-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.405776 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404590    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-daemon-config\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.405776 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404616    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-socket-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.405776 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404667    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-kubernetes\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.405776 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404689    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-run-ovn\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.405776 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.404720    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gmb7\" (UniqueName: \"kubernetes.io/projected/e62bee01-e717-4484-bb3d-ff4af522a355-kube-api-access-4gmb7\") pod \"network-metrics-daemon-q7q6c\" (UID: \"e62bee01-e717-4484-bb3d-ff4af522a355\") " pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:03.407405 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.407384    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 01:34:03.407523 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.407404    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vr5s4\""
Nov 04 01:34:03.407523 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.407386    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 01:34:03.407718 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.407619    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 01:34:03.407872 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.407855    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 01:34:03.407948 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.407917    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 01:34:03.407948 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.407931    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 01:34:03.409119 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.409102    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 01:34:03.409212 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.409198    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 01:34:03.409635 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.409613    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xwn9r\""
Nov 04 01:34:03.411839 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.411817    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 01:34:03.411920 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.411874    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 01:34:03.411920 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.411877    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-shfxh\""
Nov 04 01:34:03.411920 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.411881    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 01:34:03.412024 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.411929    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 01:34:03.417546 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.417528    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 01:34:03.417642 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.417619    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 01:34:03.454101 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.454081    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zbbgd\""
Nov 04 01:34:03.505661 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505631    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-etc-openvswitch\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.505804 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505666    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-cni-dir\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.505804 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505694    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-hostroot\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.505804 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505718    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-conf-dir\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.505804 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505735    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-etc-openvswitch\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.505804 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505739    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-run-multus-certs\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.505804 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505782    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-hostroot\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.505804 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505806    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-run-multus-certs\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.506140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505818    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20296e8e-e25f-40e1-9b2b-4971b7e69141-host\") pod \"node-ca-z992q\" (UID: \"20296e8e-e25f-40e1-9b2b-4971b7e69141\") " pod="openshift-image-registry/node-ca-z992q"
Nov 04 01:34:03.506140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505860    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-sys-fs\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.506140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505866    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-cni-dir\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.506140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505886    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-sysconfig\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.506140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505913    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20296e8e-e25f-40e1-9b2b-4971b7e69141-host\") pod \"node-ca-z992q\" (UID: \"20296e8e-e25f-40e1-9b2b-4971b7e69141\") " pod="openshift-image-registry/node-ca-z992q"
Nov 04 01:34:03.506140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505912    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-kubelet\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.506140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505939    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-socket-dir-parent\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.506140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505939    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-sys-fs\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.506140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505952    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-kubelet\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.506140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505975    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/20296e8e-e25f-40e1-9b2b-4971b7e69141-serviceca\") pod \"node-ca-z992q\" (UID: \"20296e8e-e25f-40e1-9b2b-4971b7e69141\") " pod="openshift-image-registry/node-ca-z992q"
Nov 04 01:34:03.506140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.505998    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-device-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.506140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506010    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-sysconfig\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.506140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506016    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-socket-dir-parent\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.506140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506019    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-run\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.506140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506054    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-run\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.506140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506065    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-host\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.506140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506067    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-device-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.506140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506092    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/192e2fec-fefb-4bc5-9516-ed7464963a10-env-overrides\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.506962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506108    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8f004fb-8c9c-4efb-b60d-358e32ba5120-system-cni-dir\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.506962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506117    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-host\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.506962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506127    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-var-lib-cni-multus\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.506962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506166    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-var-lib-cni-multus\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.506962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506168    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8f004fb-8c9c-4efb-b60d-358e32ba5120-system-cni-dir\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.506962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506199    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-sys\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.506962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506234    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-run-systemd\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.506962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506258    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-log-socket\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.506962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506284    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/192e2fec-fefb-4bc5-9516-ed7464963a10-ovn-node-metrics-cert\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.506962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506286    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-sys\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.506962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506307    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/67ed6df2-450d-4705-8a11-f98d5b507dc4-cni-binary-copy\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.506962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506331    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.506962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506335    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-run-systemd\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.506962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506355    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-etc-selinux\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.506962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506379    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c8rj\" (UniqueName: \"kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj\") pod \"network-check-target-hbs2c\" (UID: \"ca08235e-943c-4d8d-939d-4ceaedd9cf92\") " pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:03.506962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506400    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-systemd-units\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.506962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506421    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-run-openvswitch\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.508153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506471    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-run-ovn-kubernetes\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.508153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506495    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.508153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506518    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8f004fb-8c9c-4efb-b60d-358e32ba5120-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.508153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506530    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/20296e8e-e25f-40e1-9b2b-4971b7e69141-serviceca\") pod \"node-ca-z992q\" (UID: \"20296e8e-e25f-40e1-9b2b-4971b7e69141\") " pod="openshift-image-registry/node-ca-z992q"
Nov 04 01:34:03.508153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506550    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-daemon-config\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.508153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506581    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/192e2fec-fefb-4bc5-9516-ed7464963a10-env-overrides\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.508153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506609    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-systemd-units\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.508153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506627    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-run-openvswitch\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.508153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506676    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-socket-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.508153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506687    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-run-ovn-kubernetes\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.508153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506576    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-socket-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.508153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506691    2575 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 01:34:03.508153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506718    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-kubernetes\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.508153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506742    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-run-ovn\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.508153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506764    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gmb7\" (UniqueName: \"kubernetes.io/projected/e62bee01-e717-4484-bb3d-ff4af522a355-kube-api-access-4gmb7\") pod \"network-metrics-daemon-q7q6c\" (UID: \"e62bee01-e717-4484-bb3d-ff4af522a355\") " pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:03.508153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506787    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7647t\" (UniqueName: \"kubernetes.io/projected/a8f004fb-8c9c-4efb-b60d-358e32ba5120-kube-api-access-7647t\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.508153 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506811    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-system-cni-dir\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.508958 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506831    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8f004fb-8c9c-4efb-b60d-358e32ba5120-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.508958 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506832    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-etc-kubernetes\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.508958 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506861    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-etc-kubernetes\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.508958 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506870    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-dbus\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:03.508958 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506895    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aa398d49-e1ee-41df-8785-b765e1f35b18-hosts-file\") pod \"node-resolver-zpmgq\" (UID: \"aa398d49-e1ee-41df-8785-b765e1f35b18\") " pod="openshift-dns/node-resolver-zpmgq"
Nov 04 01:34:03.508958 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506921    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a8f004fb-8c9c-4efb-b60d-358e32ba5120-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.508958 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506931    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-kubernetes\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.508958 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506949    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:03.508958 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506965    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-run-ovn\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.508958 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506997    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-run-netns\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.508958 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507043    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-cni-bin\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.508958 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507080    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6fjs\" (UniqueName: \"kubernetes.io/projected/52cfdb11-898a-4673-97c5-ecddc1385a03-kube-api-access-m6fjs\") pod \"iptables-alerter-2sbtc\" (UID: \"52cfdb11-898a-4673-97c5-ecddc1385a03\") " pod="openshift-network-operator/iptables-alerter-2sbtc"
Nov 04 01:34:03.508958 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507105    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-tuned\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.508958 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507123    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/67ed6df2-450d-4705-8a11-f98d5b507dc4-cni-binary-copy\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.508958 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507130    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37d34756-13c3-4993-a3d1-07f68eada6b0-tmp\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.508958 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507175    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvlmf\" (UniqueName: \"kubernetes.io/projected/37d34756-13c3-4993-a3d1-07f68eada6b0-kube-api-access-xvlmf\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.508958 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507203    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-slash\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.509701 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507229    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzftg\" (UniqueName: \"kubernetes.io/projected/aa398d49-e1ee-41df-8785-b765e1f35b18-kube-api-access-lzftg\") pod \"node-resolver-zpmgq\" (UID: \"aa398d49-e1ee-41df-8785-b765e1f35b18\") " pod="openshift-dns/node-resolver-zpmgq"
Nov 04 01:34:03.509701 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507292    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.509701 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507393    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-etc-selinux\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.509701 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507467    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-system-cni-dir\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.509701 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507500    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58s5t\" (UniqueName: \"kubernetes.io/projected/20296e8e-e25f-40e1-9b2b-4971b7e69141-kube-api-access-58s5t\") pod \"node-ca-z992q\" (UID: \"20296e8e-e25f-40e1-9b2b-4971b7e69141\") " pod="openshift-image-registry/node-ca-z992q"
Nov 04 01:34:03.509701 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507523    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-var-lib-openvswitch\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.509701 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507546    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-run-k8s-cni-cncf-io\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.509701 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507568    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-var-lib-kubelet\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.509701 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507590    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-registration-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.509701 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507614    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-node-log\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.509701 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507634    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52cfdb11-898a-4673-97c5-ecddc1385a03-host-slash\") pod \"iptables-alerter-2sbtc\" (UID: \"52cfdb11-898a-4673-97c5-ecddc1385a03\") " pod="openshift-network-operator/iptables-alerter-2sbtc"
Nov 04 01:34:03.509701 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507655    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-modprobe-d\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.509701 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507676    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/480e250d-44a3-4d5c-987d-22737107544e-agent-certs\") pod \"konnectivity-agent-kwbdd\" (UID: \"480e250d-44a3-4d5c-987d-22737107544e\") " pod="kube-system/konnectivity-agent-kwbdd"
Nov 04 01:34:03.509701 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507699    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/192e2fec-fefb-4bc5-9516-ed7464963a10-ovnkube-script-lib\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.509701 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507723    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8f004fb-8c9c-4efb-b60d-358e32ba5120-cni-binary-copy\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.509701 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507745    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxtjk\" (UniqueName: \"kubernetes.io/projected/67ed6df2-450d-4705-8a11-f98d5b507dc4-kube-api-access-gxtjk\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.509701 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507776    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-lib-modules\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.510648 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507811    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-conf-dir\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.510648 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507850    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-slash\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.510648 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507824    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a8f004fb-8c9c-4efb-b60d-358e32ba5120-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.510648 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507900    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-run-netns\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.510648 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507945    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vz9t\" (UniqueName: \"kubernetes.io/projected/7acf7f6e-efb8-498e-8066-a60fbedecd49-kube-api-access-4vz9t\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.510648 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507962    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-log-socket\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.510648 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507975    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-cni-netd\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.510648 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.507999    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs\") pod \"network-metrics-daemon-q7q6c\" (UID: \"e62bee01-e717-4484-bb3d-ff4af522a355\") " pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:03.510648 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508026    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/52cfdb11-898a-4673-97c5-ecddc1385a03-iptables-alerter-script\") pod \"iptables-alerter-2sbtc\" (UID: \"52cfdb11-898a-4673-97c5-ecddc1385a03\") " pod="openshift-network-operator/iptables-alerter-2sbtc"
Nov 04 01:34:03.510648 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508037    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52cfdb11-898a-4673-97c5-ecddc1385a03-host-slash\") pod \"iptables-alerter-2sbtc\" (UID: \"52cfdb11-898a-4673-97c5-ecddc1385a03\") " pod="openshift-network-operator/iptables-alerter-2sbtc"
Nov 04 01:34:03.510648 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508049    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-var-lib-cni-bin\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.510648 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508072    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-sysctl-conf\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.510648 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508090    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-modprobe-d\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.510648 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508095    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-systemd\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.510648 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508148    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-systemd\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.510648 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508175    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-var-lib-kubelet\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.510648 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508200    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/480e250d-44a3-4d5c-987d-22737107544e-konnectivity-ca\") pod \"konnectivity-agent-kwbdd\" (UID: \"480e250d-44a3-4d5c-987d-22737107544e\") " pod="kube-system/konnectivity-agent-kwbdd"
Nov 04 01:34:03.511471 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508229    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/192e2fec-fefb-4bc5-9516-ed7464963a10-ovnkube-config\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.511471 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508255    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxtbz\" (UniqueName: \"kubernetes.io/projected/192e2fec-fefb-4bc5-9516-ed7464963a10-kube-api-access-gxtbz\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.511471 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508280    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa398d49-e1ee-41df-8785-b765e1f35b18-tmp-dir\") pod \"node-resolver-zpmgq\" (UID: \"aa398d49-e1ee-41df-8785-b765e1f35b18\") " pod="openshift-dns/node-resolver-zpmgq"
Nov 04 01:34:03.511471 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508306    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8f004fb-8c9c-4efb-b60d-358e32ba5120-cnibin\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.511471 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508317    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-run-k8s-cni-cncf-io\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.511471 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508319    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-var-lib-openvswitch\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.511471 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508329    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-cnibin\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.511471 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508369    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aa398d49-e1ee-41df-8785-b765e1f35b18-hosts-file\") pod \"node-resolver-zpmgq\" (UID: \"aa398d49-e1ee-41df-8785-b765e1f35b18\") " pod="openshift-dns/node-resolver-zpmgq"
Nov 04 01:34:03.511471 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508378    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-cnibin\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.511471 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.506896    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.511471 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508399    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-kubelet-config\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:03.511471 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508438    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-sysctl-d\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.511471 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:03.508508    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:34:03.511471 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:03.508579    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret podName:134fc8ea-90c5-4b4c-86a3-7b83d950f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:04.008547266 +0000 UTC m=+2.303154807 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret") pod "global-pull-secret-syncer-hjqq9" (UID: "134fc8ea-90c5-4b4c-86a3-7b83d950f3e5") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:34:03.511471 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508600    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/192e2fec-fefb-4bc5-9516-ed7464963a10-ovnkube-script-lib\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.511471 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508616    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-sysctl-d\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.511471 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508622    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-run-netns\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.512074 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508650    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-cni-netd\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.512074 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508674    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-var-lib-kubelet\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.512074 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:03.508692    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:34:03.512074 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:03.508732    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs podName:e62bee01-e717-4484-bb3d-ff4af522a355 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:04.008719763 +0000 UTC m=+2.303327306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs") pod "network-metrics-daemon-q7q6c" (UID: "e62bee01-e717-4484-bb3d-ff4af522a355") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:34:03.512074 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508748    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-registration-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.512074 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508774    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-cni-bin\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.512074 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508790    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-node-log\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.512074 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508218    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-daemon-config\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.512074 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509095    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-dbus\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:03.512074 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509094    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8f004fb-8c9c-4efb-b60d-358e32ba5120-cni-binary-copy\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.512074 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.508672    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-var-lib-cni-bin\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.512074 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509190    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/192e2fec-fefb-4bc5-9516-ed7464963a10-ovnkube-config\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.512074 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509193    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-var-lib-kubelet\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.512074 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509285    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-lib-modules\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.512074 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509339    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8f004fb-8c9c-4efb-b60d-358e32ba5120-cnibin\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.512074 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509412    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 01:34:03.512074 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509432    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-sysctl-conf\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.512074 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509470    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509572    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-kubelet-config\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509590    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509619    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa398d49-e1ee-41df-8785-b765e1f35b18-tmp-dir\") pod \"node-resolver-zpmgq\" (UID: \"aa398d49-e1ee-41df-8785-b765e1f35b18\") " pod="openshift-dns/node-resolver-zpmgq"
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509636    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-sj5vk\""
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509648    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509704    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-run-netns\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509716    2575 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509728    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509772    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/52cfdb11-898a-4673-97c5-ecddc1385a03-iptables-alerter-script\") pod \"iptables-alerter-2sbtc\" (UID: \"52cfdb11-898a-4673-97c5-ecddc1385a03\") " pod="openshift-network-operator/iptables-alerter-2sbtc"
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509779    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6wbxz"
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509802    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509819    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509819    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ctj2f\""
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509840    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509903    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5jndn\""
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509911    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.509912    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.510065    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tfqdd\""
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.510977    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37d34756-13c3-4993-a3d1-07f68eada6b0-tmp\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.512008    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-tuned\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.512552 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.512082    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/192e2fec-fefb-4bc5-9516-ed7464963a10-ovn-node-metrics-cert\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.520550 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.520509    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/480e250d-44a3-4d5c-987d-22737107544e-konnectivity-ca\") pod \"konnectivity-agent-kwbdd\" (UID: \"480e250d-44a3-4d5c-987d-22737107544e\") " pod="kube-system/konnectivity-agent-kwbdd"
Nov 04 01:34:03.520550 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.520536    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a8f004fb-8c9c-4efb-b60d-358e32ba5120-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.520765 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.520559    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a8f004fb-8c9c-4efb-b60d-358e32ba5120-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.521327 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.521311    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/480e250d-44a3-4d5c-987d-22737107544e-agent-certs\") pod \"konnectivity-agent-kwbdd\" (UID: \"480e250d-44a3-4d5c-987d-22737107544e\") " pod="kube-system/konnectivity-agent-kwbdd"
Nov 04 01:34:03.567513 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:03.567479    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:34:03.567513 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:03.567511    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:34:03.567513 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:03.567522    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7c8rj for pod openshift-network-diagnostics/network-check-target-hbs2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:34:03.567829 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:03.567619    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj podName:ca08235e-943c-4d8d-939d-4ceaedd9cf92 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:04.067595098 +0000 UTC m=+2.362202656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7c8rj" (UniqueName: "kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj") pod "network-check-target-hbs2c" (UID: "ca08235e-943c-4d8d-939d-4ceaedd9cf92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:34:03.570907 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.570844    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7647t\" (UniqueName: \"kubernetes.io/projected/a8f004fb-8c9c-4efb-b60d-358e32ba5120-kube-api-access-7647t\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.570907 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.570867    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvlmf\" (UniqueName: \"kubernetes.io/projected/37d34756-13c3-4993-a3d1-07f68eada6b0-kube-api-access-xvlmf\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.571078 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.570842    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gmb7\" (UniqueName: \"kubernetes.io/projected/e62bee01-e717-4484-bb3d-ff4af522a355-kube-api-access-4gmb7\") pod \"network-metrics-daemon-q7q6c\" (UID: \"e62bee01-e717-4484-bb3d-ff4af522a355\") " pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:03.571130 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.571089    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6fjs\" (UniqueName: \"kubernetes.io/projected/52cfdb11-898a-4673-97c5-ecddc1385a03-kube-api-access-m6fjs\") pod \"iptables-alerter-2sbtc\" (UID: \"52cfdb11-898a-4673-97c5-ecddc1385a03\") " pod="openshift-network-operator/iptables-alerter-2sbtc"
Nov 04 01:34:03.571170 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.571156    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzftg\" (UniqueName: \"kubernetes.io/projected/aa398d49-e1ee-41df-8785-b765e1f35b18-kube-api-access-lzftg\") pod \"node-resolver-zpmgq\" (UID: \"aa398d49-e1ee-41df-8785-b765e1f35b18\") " pod="openshift-dns/node-resolver-zpmgq"
Nov 04 01:34:03.572078 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.572052    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58s5t\" (UniqueName: \"kubernetes.io/projected/20296e8e-e25f-40e1-9b2b-4971b7e69141-kube-api-access-58s5t\") pod \"node-ca-z992q\" (UID: \"20296e8e-e25f-40e1-9b2b-4971b7e69141\") " pod="openshift-image-registry/node-ca-z992q"
Nov 04 01:34:03.572173 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.572100    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxtjk\" (UniqueName: \"kubernetes.io/projected/67ed6df2-450d-4705-8a11-f98d5b507dc4-kube-api-access-gxtjk\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.572377 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.572360    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vz9t\" (UniqueName: \"kubernetes.io/projected/7acf7f6e-efb8-498e-8066-a60fbedecd49-kube-api-access-4vz9t\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.572718 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.572701    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxtbz\" (UniqueName: \"kubernetes.io/projected/192e2fec-fefb-4bc5-9516-ed7464963a10-kube-api-access-gxtbz\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.688599 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.688566    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2sbtc"
Nov 04 01:34:03.724990 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.724958    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:03.727938 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.727913    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zpmgq"
Nov 04 01:34:03.730194 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.730177    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hcvld"
Nov 04 01:34:03.739831 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.739807    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z992q"
Nov 04 01:34:03.768109 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.768067    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:34:03.768993 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:03.768959    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d9a5980aefdf4e226d40970221f0eb6.slice/crio-5471b6fa46f5222a9d35b493eaece92ee03c1f1d0b93043685e526dfe5784179 WatchSource:0}: Error finding container 5471b6fa46f5222a9d35b493eaece92ee03c1f1d0b93043685e526dfe5784179: Status 404 returned error can't find the container with id 5471b6fa46f5222a9d35b493eaece92ee03c1f1d0b93043685e526dfe5784179
Nov 04 01:34:03.769214 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:03.769189    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa398d49_e1ee_41df_8785_b765e1f35b18.slice/crio-2f3c96e1085f141b75bcef65fb3d5f5f9605cfd40e3718e1989ec632be023f3e WatchSource:0}: Error finding container 2f3c96e1085f141b75bcef65fb3d5f5f9605cfd40e3718e1989ec632be023f3e: Status 404 returned error can't find the container with id 2f3c96e1085f141b75bcef65fb3d5f5f9605cfd40e3718e1989ec632be023f3e
Nov 04 01:34:03.769946 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:03.769926    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod192e2fec_fefb_4bc5_9516_ed7464963a10.slice/crio-8c41e311573c0941f5600843942d61c63ceeece2bbd20c365768575c3b2430f8 WatchSource:0}: Error finding container 8c41e311573c0941f5600843942d61c63ceeece2bbd20c365768575c3b2430f8: Status 404 returned error can't find the container with id 8c41e311573c0941f5600843942d61c63ceeece2bbd20c365768575c3b2430f8
Nov 04 01:34:03.771039 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:03.771013    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20296e8e_e25f_40e1_9b2b_4971b7e69141.slice/crio-2a3a6f723c4544bbd5e53e1828f49093eda93bb90a92e7449075831c3392917d WatchSource:0}: Error finding container 2a3a6f723c4544bbd5e53e1828f49093eda93bb90a92e7449075831c3392917d: Status 404 returned error can't find the container with id 2a3a6f723c4544bbd5e53e1828f49093eda93bb90a92e7449075831c3392917d
Nov 04 01:34:03.771794 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:03.771769    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc58b01dca1916bc539c28fce5659b181.slice/crio-84fb8fd6e9424f9f7ce12f856c19373e6bee5a388fedaa3eabd1636fa96c6389 WatchSource:0}: Error finding container 84fb8fd6e9424f9f7ce12f856c19373e6bee5a388fedaa3eabd1636fa96c6389: Status 404 returned error can't find the container with id 84fb8fd6e9424f9f7ce12f856c19373e6bee5a388fedaa3eabd1636fa96c6389
Nov 04 01:34:03.772555 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:03.772529    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67ed6df2_450d_4705_8a11_f98d5b507dc4.slice/crio-74b1b2138f9f936a36d0e5a89ca49ec63b1354e0f13379fdb892bdc26b16b2c5 WatchSource:0}: Error finding container 74b1b2138f9f936a36d0e5a89ca49ec63b1354e0f13379fdb892bdc26b16b2c5: Status 404 returned error can't find the container with id 74b1b2138f9f936a36d0e5a89ca49ec63b1354e0f13379fdb892bdc26b16b2c5
Nov 04 01:34:03.776343 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.776308    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:34:03.776904 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.776880    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:34:03.787306 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:03.787278    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7acf7f6e_efb8_498e_8066_a60fbedecd49.slice/crio-52aacf3160fabbfbae094dfd665f1b3e17861bd327f8330ff8be7902ee822456 WatchSource:0}: Error finding container 52aacf3160fabbfbae094dfd665f1b3e17861bd327f8330ff8be7902ee822456: Status 404 returned error can't find the container with id 52aacf3160fabbfbae094dfd665f1b3e17861bd327f8330ff8be7902ee822456
Nov 04 01:34:03.791574 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.791555    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:34:03.796348 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:03.796331    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kwbdd"
Nov 04 01:34:03.801126 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:03.801102    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37d34756_13c3_4993_a3d1_07f68eada6b0.slice/crio-be7aa8d5e548488904d64872c0b3e7dbf4494984000bd1b95a4ca18bd8ebbe07 WatchSource:0}: Error finding container be7aa8d5e548488904d64872c0b3e7dbf4494984000bd1b95a4ca18bd8ebbe07: Status 404 returned error can't find the container with id be7aa8d5e548488904d64872c0b3e7dbf4494984000bd1b95a4ca18bd8ebbe07
Nov 04 01:34:03.805169 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:03.805147    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod480e250d_44a3_4d5c_987d_22737107544e.slice/crio-3f0fb6f9a3020f5743953f382e19d597cb410dba4214a2216fc9faf51dfab262 WatchSource:0}: Error finding container 3f0fb6f9a3020f5743953f382e19d597cb410dba4214a2216fc9faf51dfab262: Status 404 returned error can't find the container with id 3f0fb6f9a3020f5743953f382e19d597cb410dba4214a2216fc9faf51dfab262
Nov 04 01:34:04.012076 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:04.012034    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs\") pod \"network-metrics-daemon-q7q6c\" (UID: \"e62bee01-e717-4484-bb3d-ff4af522a355\") " pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:04.012265 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:04.012107    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:04.012265 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:04.012199    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:34:04.012265 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:04.012208    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:34:04.012361 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:04.012267    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret podName:134fc8ea-90c5-4b4c-86a3-7b83d950f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:05.012249306 +0000 UTC m=+3.306856848 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret") pod "global-pull-secret-syncer-hjqq9" (UID: "134fc8ea-90c5-4b4c-86a3-7b83d950f3e5") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:34:04.012361 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:04.012291    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs podName:e62bee01-e717-4484-bb3d-ff4af522a355 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:05.012282828 +0000 UTC m=+3.306890370 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs") pod "network-metrics-daemon-q7q6c" (UID: "e62bee01-e717-4484-bb3d-ff4af522a355") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:34:04.112808 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:04.112715    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c8rj\" (UniqueName: \"kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj\") pod \"network-check-target-hbs2c\" (UID: \"ca08235e-943c-4d8d-939d-4ceaedd9cf92\") " pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:04.113412 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:04.112886    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:34:04.113412 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:04.112907    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:34:04.113412 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:04.112917    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7c8rj for pod openshift-network-diagnostics/network-check-target-hbs2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:34:04.113412 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:04.112970    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj podName:ca08235e-943c-4d8d-939d-4ceaedd9cf92 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:05.112956147 +0000 UTC m=+3.407563689 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7c8rj" (UniqueName: "kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj") pod "network-check-target-hbs2c" (UID: "ca08235e-943c-4d8d-939d-4ceaedd9cf92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:34:04.255347 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:04.255291    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kwbdd" event={"ID":"480e250d-44a3-4d5c-987d-22737107544e","Type":"ContainerStarted","Data":"3f0fb6f9a3020f5743953f382e19d597cb410dba4214a2216fc9faf51dfab262"}
Nov 04 01:34:04.256563 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:04.256536    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hhttg" event={"ID":"37d34756-13c3-4993-a3d1-07f68eada6b0","Type":"ContainerStarted","Data":"be7aa8d5e548488904d64872c0b3e7dbf4494984000bd1b95a4ca18bd8ebbe07"}
Nov 04 01:34:04.257682 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:04.257656    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx" event={"ID":"7acf7f6e-efb8-498e-8066-a60fbedecd49","Type":"ContainerStarted","Data":"52aacf3160fabbfbae094dfd665f1b3e17861bd327f8330ff8be7902ee822456"}
Nov 04 01:34:04.258757 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:04.258738    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hh4tb" event={"ID":"a8f004fb-8c9c-4efb-b60d-358e32ba5120","Type":"ContainerStarted","Data":"6f74ad59227901ed5d82dbb766f8ab4258717c3beb210c4cfa68d3b575b27ec3"}
Nov 04 01:34:04.259855 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:04.259834    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z992q" event={"ID":"20296e8e-e25f-40e1-9b2b-4971b7e69141","Type":"ContainerStarted","Data":"2a3a6f723c4544bbd5e53e1828f49093eda93bb90a92e7449075831c3392917d"}
Nov 04 01:34:04.260981 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:04.260960    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hcvld" event={"ID":"67ed6df2-450d-4705-8a11-f98d5b507dc4","Type":"ContainerStarted","Data":"74b1b2138f9f936a36d0e5a89ca49ec63b1354e0f13379fdb892bdc26b16b2c5"}
Nov 04 01:34:04.261984 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:04.261966    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal" event={"ID":"c58b01dca1916bc539c28fce5659b181","Type":"ContainerStarted","Data":"84fb8fd6e9424f9f7ce12f856c19373e6bee5a388fedaa3eabd1636fa96c6389"}
Nov 04 01:34:04.263032 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:04.263013    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zpmgq" event={"ID":"aa398d49-e1ee-41df-8785-b765e1f35b18","Type":"ContainerStarted","Data":"2f3c96e1085f141b75bcef65fb3d5f5f9605cfd40e3718e1989ec632be023f3e"}
Nov 04 01:34:04.264487 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:04.264449    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal" event={"ID":"5d9a5980aefdf4e226d40970221f0eb6","Type":"ContainerStarted","Data":"5471b6fa46f5222a9d35b493eaece92ee03c1f1d0b93043685e526dfe5784179"}
Nov 04 01:34:04.265603 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:04.265583    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv" event={"ID":"192e2fec-fefb-4bc5-9516-ed7464963a10","Type":"ContainerStarted","Data":"8c41e311573c0941f5600843942d61c63ceeece2bbd20c365768575c3b2430f8"}
Nov 04 01:34:04.452226 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:04.452189    2575 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:34:04.462119 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:04.462095    2575 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:34:04.514108 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:04.514056    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 01:29:03 +0000 UTC" deadline="2028-07-14 21:39:26.440601872 +0000 UTC"
Nov 04 01:34:04.514108 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:04.514101    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14852h5m21.926503857s"
Nov 04 01:34:04.543863 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:04.543824    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52cfdb11_898a_4673_97c5_ecddc1385a03.slice/crio-c9933ed453da9a8ee31ec474c134862f5d132629cfd5b1f20aca266bcd74b75a WatchSource:0}: Error finding container c9933ed453da9a8ee31ec474c134862f5d132629cfd5b1f20aca266bcd74b75a: Status 404 returned error can't find the container with id c9933ed453da9a8ee31ec474c134862f5d132629cfd5b1f20aca266bcd74b75a
Nov 04 01:34:04.665887 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:04.665849    2575 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:34:05.019079 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:05.019043    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:05.019300 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:05.019093    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs\") pod \"network-metrics-daemon-q7q6c\" (UID: \"e62bee01-e717-4484-bb3d-ff4af522a355\") " pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:05.019300 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:05.019245    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:34:05.019412 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:05.019304    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs podName:e62bee01-e717-4484-bb3d-ff4af522a355 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:07.019284381 +0000 UTC m=+5.313891924 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs") pod "network-metrics-daemon-q7q6c" (UID: "e62bee01-e717-4484-bb3d-ff4af522a355") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:34:05.019412 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:05.019346    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:34:05.019412 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:05.019368    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret podName:134fc8ea-90c5-4b4c-86a3-7b83d950f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:07.01936114 +0000 UTC m=+5.313968683 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret") pod "global-pull-secret-syncer-hjqq9" (UID: "134fc8ea-90c5-4b4c-86a3-7b83d950f3e5") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:34:05.121892 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:05.121845    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c8rj\" (UniqueName: \"kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj\") pod \"network-check-target-hbs2c\" (UID: \"ca08235e-943c-4d8d-939d-4ceaedd9cf92\") " pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:05.122634 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:05.122089    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:34:05.122634 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:05.122111    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:34:05.122634 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:05.122124    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7c8rj for pod openshift-network-diagnostics/network-check-target-hbs2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:34:05.122634 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:05.122188    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj podName:ca08235e-943c-4d8d-939d-4ceaedd9cf92 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:07.122169931 +0000 UTC m=+5.416777477 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7c8rj" (UniqueName: "kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj") pod "network-check-target-hbs2c" (UID: "ca08235e-943c-4d8d-939d-4ceaedd9cf92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:34:05.251096 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:05.251053    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:05.251308 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:05.251222    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:05.251388 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:05.251321    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:05.251444 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:05.251393    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:05.251502 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:05.251445    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:05.251588 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:05.251526    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:05.287775 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:05.287677    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2sbtc" event={"ID":"52cfdb11-898a-4673-97c5-ecddc1385a03","Type":"ContainerStarted","Data":"c9933ed453da9a8ee31ec474c134862f5d132629cfd5b1f20aca266bcd74b75a"}
Nov 04 01:34:05.515135 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:05.515034    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 01:29:03 +0000 UTC" deadline="2028-05-11 16:12:47.267014995 +0000 UTC"
Nov 04 01:34:05.515135 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:05.515101    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13310h38m41.75191844s"
Nov 04 01:34:07.039694 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:07.039655    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs\") pod \"network-metrics-daemon-q7q6c\" (UID: \"e62bee01-e717-4484-bb3d-ff4af522a355\") " pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:07.040187 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:07.039749    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:07.040187 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:07.039887    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:34:07.040187 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:07.039952    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret podName:134fc8ea-90c5-4b4c-86a3-7b83d950f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:11.039934253 +0000 UTC m=+9.334541810 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret") pod "global-pull-secret-syncer-hjqq9" (UID: "134fc8ea-90c5-4b4c-86a3-7b83d950f3e5") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:34:07.040187 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:07.040031    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:34:07.040187 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:07.040066    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs podName:e62bee01-e717-4484-bb3d-ff4af522a355 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:11.04005402 +0000 UTC m=+9.334661578 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs") pod "network-metrics-daemon-q7q6c" (UID: "e62bee01-e717-4484-bb3d-ff4af522a355") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:34:07.141053 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:07.140920    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c8rj\" (UniqueName: \"kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj\") pod \"network-check-target-hbs2c\" (UID: \"ca08235e-943c-4d8d-939d-4ceaedd9cf92\") " pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:07.141227 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:07.141136    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:34:07.141227 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:07.141163    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:34:07.141227 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:07.141180    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7c8rj for pod openshift-network-diagnostics/network-check-target-hbs2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:34:07.141449 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:07.141252    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj podName:ca08235e-943c-4d8d-939d-4ceaedd9cf92 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:11.141232996 +0000 UTC m=+9.435840554 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7c8rj" (UniqueName: "kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj") pod "network-check-target-hbs2c" (UID: "ca08235e-943c-4d8d-939d-4ceaedd9cf92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:34:07.250988 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:07.250532    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:07.250988 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:07.250678    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:07.250988 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:07.250752    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:07.250988 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:07.250837    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:07.250988 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:07.250882    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:07.250988 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:07.250948    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:09.251075 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:09.250793    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:09.251075 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:09.251064    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:09.251794 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:09.251000    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:09.251794 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:09.251187    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:09.251794 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:09.251350    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:09.251794 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:09.251398    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:09.324366 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:09.323632    2575 generic.go:356] "Generic (PLEG): container finished" podID="a8f004fb-8c9c-4efb-b60d-358e32ba5120" containerID="a451e1ac3a8bbb90ce38288c508334767da67aa3b32bd20f28f505988eb2372a" exitCode=0
Nov 04 01:34:09.324366 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:09.323748    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hh4tb" event={"ID":"a8f004fb-8c9c-4efb-b60d-358e32ba5120","Type":"ContainerDied","Data":"a451e1ac3a8bbb90ce38288c508334767da67aa3b32bd20f28f505988eb2372a"}
Nov 04 01:34:09.332212 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:09.330312    2575 generic.go:356] "Generic (PLEG): container finished" podID="c58b01dca1916bc539c28fce5659b181" containerID="bf475d1803d2ac5df8f1e3bbca0232a0e67e021a6ddb6fa85997dddb2ade8527" exitCode=0
Nov 04 01:34:09.332212 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:09.330362    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal" event={"ID":"c58b01dca1916bc539c28fce5659b181","Type":"ContainerDied","Data":"bf475d1803d2ac5df8f1e3bbca0232a0e67e021a6ddb6fa85997dddb2ade8527"}
Nov 04 01:34:10.349967 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:10.349926    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kwbdd" event={"ID":"480e250d-44a3-4d5c-987d-22737107544e","Type":"ContainerStarted","Data":"9c903f00fcbdc68e41e07c5c8748841dead7a88477c9b500db6976463c33a61d"}
Nov 04 01:34:10.355960 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:10.355624    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal" event={"ID":"c58b01dca1916bc539c28fce5659b181","Type":"ContainerStarted","Data":"9cd1628279a72a1a5bdad88e54a4451d55f29753bb8a2ddc91d63b3d155281ec"}
Nov 04 01:34:10.358520 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:10.358489    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal" event={"ID":"5d9a5980aefdf4e226d40970221f0eb6","Type":"ContainerStarted","Data":"7ecf8fc37ef1fc332541c023e68a53059b0f5c2b9c1bb2742442b65ef023d0f5"}
Nov 04 01:34:10.468440 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:10.468373    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-kwbdd" podStartSLOduration=3.091348699 podStartE2EDuration="8.468353303s" totalImagesPullingTime="5.377004604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:10.467211472 +0000 UTC m=+8.761819035" watchObservedRunningTime="2026-11-04 01:34:10.468353303 +0000 UTC m=+8.762960866"
Nov 04 01:34:10.586581 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:10.586441    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal" podStartSLOduration=8.586426157 podStartE2EDuration="8.586426157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:34:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:10.58558443 +0000 UTC m=+8.880191997" watchObservedRunningTime="2026-11-04 01:34:10.586426157 +0000 UTC m=+8.881033718"
Nov 04 01:34:11.079072 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:11.079037    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:11.079246 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:11.079090    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs\") pod \"network-metrics-daemon-q7q6c\" (UID: \"e62bee01-e717-4484-bb3d-ff4af522a355\") " pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:11.079246 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:11.079225    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:34:11.079357 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:11.079280    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs podName:e62bee01-e717-4484-bb3d-ff4af522a355 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:19.079264117 +0000 UTC m=+17.373871675 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs") pod "network-metrics-daemon-q7q6c" (UID: "e62bee01-e717-4484-bb3d-ff4af522a355") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:34:11.079357 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:11.079324    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:34:11.079357 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:11.079351    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret podName:134fc8ea-90c5-4b4c-86a3-7b83d950f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:19.079343701 +0000 UTC m=+17.373951256 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret") pod "global-pull-secret-syncer-hjqq9" (UID: "134fc8ea-90c5-4b4c-86a3-7b83d950f3e5") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:34:11.180398 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:11.180358    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c8rj\" (UniqueName: \"kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj\") pod \"network-check-target-hbs2c\" (UID: \"ca08235e-943c-4d8d-939d-4ceaedd9cf92\") " pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:11.180606 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:11.180549    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:34:11.180606 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:11.180570    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:34:11.180606 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:11.180584    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7c8rj for pod openshift-network-diagnostics/network-check-target-hbs2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:34:11.180727 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:11.180639    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj podName:ca08235e-943c-4d8d-939d-4ceaedd9cf92 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:19.180621755 +0000 UTC m=+17.475229309 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7c8rj" (UniqueName: "kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj") pod "network-check-target-hbs2c" (UID: "ca08235e-943c-4d8d-939d-4ceaedd9cf92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:34:11.250375 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:11.250330    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:11.250558 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:11.250500    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:11.250637 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:11.250572    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:11.250637 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:11.250586    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:11.250734 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:11.250685    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:11.250788 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:11.250757    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:11.364735 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:11.364697    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx" event={"ID":"7acf7f6e-efb8-498e-8066-a60fbedecd49","Type":"ContainerStarted","Data":"dee5331fe1af279e481d9af3408cbdb1a0c30f5377e76bbe2dc1f0d1f9120fe0"}
Nov 04 01:34:11.368811 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:11.368730    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z992q" event={"ID":"20296e8e-e25f-40e1-9b2b-4971b7e69141","Type":"ContainerStarted","Data":"5086079edf142c026aefc1b826e2ec68f1648abd3a3cd823131428f8e298dfaa"}
Nov 04 01:34:11.487381 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:11.486778    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal" podStartSLOduration=8.486757038 podStartE2EDuration="8.486757038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:34:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:10.701835752 +0000 UTC m=+8.996443317" watchObservedRunningTime="2026-11-04 01:34:11.486757038 +0000 UTC m=+9.781364603"
Nov 04 01:34:11.487381 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:11.487151    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-z992q" podStartSLOduration=2.999781903 podStartE2EDuration="9.487137951s" totalImagesPullingTime="6.487356048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:11.485989839 +0000 UTC m=+9.780597427" watchObservedRunningTime="2026-11-04 01:34:11.487137951 +0000 UTC m=+9.781745515"
Nov 04 01:34:12.906518 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:12.906478    2575 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 01:34:13.181345 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:13.181227    2575 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T01:34:12.906516129Z","UUID":"d6fba758-084b-4ee9-a0d2-ae872a490593","Handler":null,"Name":"","Endpoint":""}
Nov 04 01:34:13.182756 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:13.182733    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 01:34:13.182862 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:13.182766    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 01:34:13.251375 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:13.251334    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:13.251573 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:13.251484    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:13.251641 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:13.251576    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:13.251690 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:13.251656    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:13.251740 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:13.251708    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:13.251787 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:13.251773    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:13.375054 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:13.374972    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hcvld" event={"ID":"67ed6df2-450d-4705-8a11-f98d5b507dc4","Type":"ContainerStarted","Data":"92d830359b7c303605203fdd5d24548bfb623c31811418c26f69de439ae8dc34"}
Nov 04 01:34:13.377870 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:13.377372    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zpmgq" event={"ID":"aa398d49-e1ee-41df-8785-b765e1f35b18","Type":"ContainerStarted","Data":"0dac85d88c395283b03b4f5b8ec90fa538a8faf2cdde509750ee5d164c1f77f0"}
Nov 04 01:34:13.381052 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:13.380975    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx" event={"ID":"7acf7f6e-efb8-498e-8066-a60fbedecd49","Type":"ContainerStarted","Data":"98749ac0b9d6d688619d889867525d941fa3371b8e62ca4e8f1450f9d3f3ccbb"}
Nov 04 01:34:13.497079 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:13.497017    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-hcvld" podStartSLOduration=2.922137868 podStartE2EDuration="11.496998581s" totalImagesPullingTime="8.574860713s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:13.496311544 +0000 UTC m=+11.790919106" watchObservedRunningTime="2026-11-04 01:34:13.496998581 +0000 UTC m=+11.791606145"
Nov 04 01:34:13.614920 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:13.614194    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-zpmgq" podStartSLOduration=2.401566566 podStartE2EDuration="11.614174552s" totalImagesPullingTime="9.212607986s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:13.613641787 +0000 UTC m=+11.908249350" watchObservedRunningTime="2026-11-04 01:34:13.614174552 +0000 UTC m=+11.908782115"
Nov 04 01:34:14.086680 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:14.086395    2575 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-kwbdd"
Nov 04 01:34:14.087930 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:14.087799    2575 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-kwbdd"
Nov 04 01:34:14.387088 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:14.386120    2575 generic.go:356] "Generic (PLEG): container finished" podID="a8f004fb-8c9c-4efb-b60d-358e32ba5120" containerID="b8a071b045d441da7f8a5c7fca9e9ede664e33494109085586489e7500ce7c6d" exitCode=0
Nov 04 01:34:14.387088 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:14.386516    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hh4tb" event={"ID":"a8f004fb-8c9c-4efb-b60d-358e32ba5120","Type":"ContainerDied","Data":"b8a071b045d441da7f8a5c7fca9e9ede664e33494109085586489e7500ce7c6d"}
Nov 04 01:34:14.387088 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:14.386976    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-kwbdd"
Nov 04 01:34:14.387309 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:14.387250    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kwbdd"
Nov 04 01:34:15.251137 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:15.251083    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:15.251628 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:15.251238    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:15.251628 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:15.251070    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:15.251628 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:15.251337    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:15.251628 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:15.251100    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:15.251628 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:15.251408    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:15.391149 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:15.391113    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx" event={"ID":"7acf7f6e-efb8-498e-8066-a60fbedecd49","Type":"ContainerStarted","Data":"37ae5d14ab25875319ea5a5d5b0a84538c2ca71d3d43e6455b68e7c399c6f333"}
Nov 04 01:34:15.392710 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:15.392681    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2sbtc" event={"ID":"52cfdb11-898a-4673-97c5-ecddc1385a03","Type":"ContainerStarted","Data":"326fac7a95fd7f2aaa03d3df62e6f9c5ae2aaa7b926a2d7672dab2006f40fc0a"}
Nov 04 01:34:15.511847 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:15.511796    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx" podStartSLOduration=3.32420909 podStartE2EDuration="13.511782567s" totalImagesPullingTime="10.187573477s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:34:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:15.511255144 +0000 UTC m=+13.805862709" watchObservedRunningTime="2026-11-04 01:34:15.511782567 +0000 UTC m=+13.806390150"
Nov 04 01:34:15.627819 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:15.627765    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2sbtc" podStartSLOduration=4.869715914 podStartE2EDuration="13.627750025s" totalImagesPullingTime="8.758034111s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:15.627626023 +0000 UTC m=+13.922233584" watchObservedRunningTime="2026-11-04 01:34:15.627750025 +0000 UTC m=+13.922357589"
Nov 04 01:34:16.397258 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:16.397209    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hhttg" event={"ID":"37d34756-13c3-4993-a3d1-07f68eada6b0","Type":"ContainerStarted","Data":"c22e93c511e358976fef7d73da34d8f31c14f2fdc21f4a22190084cae058b5c8"}
Nov 04 01:34:16.399721 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:16.399694    2575 generic.go:356] "Generic (PLEG): container finished" podID="a8f004fb-8c9c-4efb-b60d-358e32ba5120" containerID="ebc9ff9a537320587ec7c17c2342bc8ef2f02c8d8be9c51f7b028a42050df65e" exitCode=0
Nov 04 01:34:16.399857 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:16.399744    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hh4tb" event={"ID":"a8f004fb-8c9c-4efb-b60d-358e32ba5120","Type":"ContainerDied","Data":"ebc9ff9a537320587ec7c17c2342bc8ef2f02c8d8be9c51f7b028a42050df65e"}
Nov 04 01:34:16.517423 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:16.517371    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hhttg" podStartSLOduration=2.845402264 podStartE2EDuration="14.517359205s" totalImagesPullingTime="11.671956941s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:16.517231501 +0000 UTC m=+14.811839062" watchObservedRunningTime="2026-11-04 01:34:16.517359205 +0000 UTC m=+14.811966766"
Nov 04 01:34:17.250312 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:17.250283    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:17.250528 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:17.250416    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:17.250528 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:17.250508    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:17.250657 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:17.250578    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:17.250657 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:17.250636    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:17.250754 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:17.250702    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:18.407383 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:18.407346    2575 generic.go:356] "Generic (PLEG): container finished" podID="a8f004fb-8c9c-4efb-b60d-358e32ba5120" containerID="cfbd049c9dfc4f539f39f9bc4b86591d4a16d2123c9f9e50c8d48d1f0778386b" exitCode=0
Nov 04 01:34:18.407909 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:18.407402    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hh4tb" event={"ID":"a8f004fb-8c9c-4efb-b60d-358e32ba5120","Type":"ContainerDied","Data":"cfbd049c9dfc4f539f39f9bc4b86591d4a16d2123c9f9e50c8d48d1f0778386b"}
Nov 04 01:34:19.143884 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:19.143667    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:19.144006 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:19.143922    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs\") pod \"network-metrics-daemon-q7q6c\" (UID: \"e62bee01-e717-4484-bb3d-ff4af522a355\") " pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:19.144006 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:19.143802    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:34:19.144006 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:19.144003    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret podName:134fc8ea-90c5-4b4c-86a3-7b83d950f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:35.143988519 +0000 UTC m=+33.438596059 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret") pod "global-pull-secret-syncer-hjqq9" (UID: "134fc8ea-90c5-4b4c-86a3-7b83d950f3e5") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:34:19.144150 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:19.144035    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:34:19.144150 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:19.144077    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs podName:e62bee01-e717-4484-bb3d-ff4af522a355 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:35.144067038 +0000 UTC m=+33.438674579 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs") pod "network-metrics-daemon-q7q6c" (UID: "e62bee01-e717-4484-bb3d-ff4af522a355") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:34:19.244247 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:19.244221    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c8rj\" (UniqueName: \"kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj\") pod \"network-check-target-hbs2c\" (UID: \"ca08235e-943c-4d8d-939d-4ceaedd9cf92\") " pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:19.244418 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:19.244400    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:34:19.244513 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:19.244425    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:34:19.244513 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:19.244439    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7c8rj for pod openshift-network-diagnostics/network-check-target-hbs2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:34:19.244589 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:19.244517    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj podName:ca08235e-943c-4d8d-939d-4ceaedd9cf92 nodeName:}" failed. No retries permitted until 2026-11-04 01:34:35.244499055 +0000 UTC m=+33.539106601 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7c8rj" (UniqueName: "kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj") pod "network-check-target-hbs2c" (UID: "ca08235e-943c-4d8d-939d-4ceaedd9cf92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:34:19.250376 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:19.250346    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:19.250376 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:19.250355    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:19.250560 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:19.250378    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:19.250560 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:19.250471    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:19.250560 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:19.250534    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:19.250698 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:19.250605    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:19.413732 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:19.413710    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6lfbv_192e2fec-fefb-4bc5-9516-ed7464963a10/ovn-acl-logging/0.log"
Nov 04 01:34:19.414593 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:19.414565    2575 generic.go:356] "Generic (PLEG): container finished" podID="192e2fec-fefb-4bc5-9516-ed7464963a10" containerID="ac8621fa6abe76d4d71b32912b440560bb522bdf5eb5471572fbc94ca4518f2a" exitCode=1
Nov 04 01:34:19.414694 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:19.414623    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv" event={"ID":"192e2fec-fefb-4bc5-9516-ed7464963a10","Type":"ContainerStarted","Data":"eada322023ba368b48e47ec367038a4ca2a03894305f7a8b1090d14f45eae6f9"}
Nov 04 01:34:19.414694 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:19.414681    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv" event={"ID":"192e2fec-fefb-4bc5-9516-ed7464963a10","Type":"ContainerStarted","Data":"76d12013bf22c926559bde38c11bf951ed8578e8f6fb72e5911ac6777eb5eb95"}
Nov 04 01:34:19.414758 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:19.414708    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv" event={"ID":"192e2fec-fefb-4bc5-9516-ed7464963a10","Type":"ContainerStarted","Data":"b4bbb669d68171911fd62ca6ddc751c08cea413a9d312986c8fea5694ce0f3f7"}
Nov 04 01:34:19.414758 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:19.414738    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv" event={"ID":"192e2fec-fefb-4bc5-9516-ed7464963a10","Type":"ContainerDied","Data":"ac8621fa6abe76d4d71b32912b440560bb522bdf5eb5471572fbc94ca4518f2a"}
Nov 04 01:34:19.414831 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:19.414765    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv" event={"ID":"192e2fec-fefb-4bc5-9516-ed7464963a10","Type":"ContainerStarted","Data":"4e10f4c9848caccdafb87541aac2c10578064fc7e1fd136d4eaa96e89abb4543"}
Nov 04 01:34:20.419323 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:20.419162    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6lfbv_192e2fec-fefb-4bc5-9516-ed7464963a10/ovn-acl-logging/0.log"
Nov 04 01:34:20.419765 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:20.419729    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv" event={"ID":"192e2fec-fefb-4bc5-9516-ed7464963a10","Type":"ContainerStarted","Data":"5e9127fd68114fd59844feabc28b56bbdc320d98d53a828420c09254b34636d4"}
Nov 04 01:34:20.422147 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:20.422119    2575 generic.go:356] "Generic (PLEG): container finished" podID="a8f004fb-8c9c-4efb-b60d-358e32ba5120" containerID="db52c6b2e59ed417d5e9f3b78599a8c83870ae017e010039390bac5a8766ce3e" exitCode=0
Nov 04 01:34:20.422248 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:20.422171    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hh4tb" event={"ID":"a8f004fb-8c9c-4efb-b60d-358e32ba5120","Type":"ContainerDied","Data":"db52c6b2e59ed417d5e9f3b78599a8c83870ae017e010039390bac5a8766ce3e"}
Nov 04 01:34:21.250364 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:21.250330    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:21.250555 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:21.250370    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:21.250555 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:21.250331    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:21.250555 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:21.250439    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:21.250672 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:21.250563    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:21.250703 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:21.250673    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:21.426858 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:21.426828    2575 generic.go:356] "Generic (PLEG): container finished" podID="a8f004fb-8c9c-4efb-b60d-358e32ba5120" containerID="e42bb43a71c16cce015d035a2457940e5a8fa26844c07625808923eae11153d9" exitCode=0
Nov 04 01:34:21.426858 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:21.426864    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hh4tb" event={"ID":"a8f004fb-8c9c-4efb-b60d-358e32ba5120","Type":"ContainerDied","Data":"e42bb43a71c16cce015d035a2457940e5a8fa26844c07625808923eae11153d9"}
Nov 04 01:34:22.431526 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:22.431487    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hh4tb" event={"ID":"a8f004fb-8c9c-4efb-b60d-358e32ba5120","Type":"ContainerStarted","Data":"027894b9e8cbef90e5f7b2253610234c604e90f343fbe1f6a292daf4f67eeb71"}
Nov 04 01:34:22.436865 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:22.436843    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6lfbv_192e2fec-fefb-4bc5-9516-ed7464963a10/ovn-acl-logging/0.log"
Nov 04 01:34:22.437241 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:22.437206    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv" event={"ID":"192e2fec-fefb-4bc5-9516-ed7464963a10","Type":"ContainerStarted","Data":"116d350a9fcc1ec8d7cf1dd5fed155cee97020078ae8ded936b751cb1be7a155"}
Nov 04 01:34:22.556463 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:22.556408    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hh4tb" podStartSLOduration=7.818406418 podStartE2EDuration="20.55639739s" totalImagesPullingTime="12.737990972s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:34:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:22.556114095 +0000 UTC m=+20.850721658" watchObservedRunningTime="2026-11-04 01:34:22.55639739 +0000 UTC m=+20.851004952"
Nov 04 01:34:23.250656 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:23.250626    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:23.250813 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:23.250626    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:23.250813 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:23.250725    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:23.250813 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:23.250631    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:23.250813 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:23.250796    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:23.250971 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:23.250862    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:24.445155 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:24.444990    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6lfbv_192e2fec-fefb-4bc5-9516-ed7464963a10/ovn-acl-logging/0.log"
Nov 04 01:34:24.445714 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:24.445443    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv" event={"ID":"192e2fec-fefb-4bc5-9516-ed7464963a10","Type":"ContainerStarted","Data":"7fe8bc2e586d12d699e90e5fbd1d3a8ed2df0eb1605d909b5a5242fe13518d92"}
Nov 04 01:34:24.445771 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:24.445732    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:24.445863 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:24.445847    2575 scope.go:122] "RemoveContainer" containerID="ac8621fa6abe76d4d71b32912b440560bb522bdf5eb5471572fbc94ca4518f2a"
Nov 04 01:34:24.462515 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:24.462493    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:25.251348 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:25.251314    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:25.251348 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:25.251340    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:25.251657 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:25.251420    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:25.251657 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:25.251548    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:25.251657 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:25.251644    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:25.251747 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:25.251700    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:25.450909 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:25.450883    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6lfbv_192e2fec-fefb-4bc5-9516-ed7464963a10/ovn-acl-logging/0.log"
Nov 04 01:34:25.451331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:25.451241    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv" event={"ID":"192e2fec-fefb-4bc5-9516-ed7464963a10","Type":"ContainerStarted","Data":"9254bc26ff8566911781bebadb78004e573010fa6360d18efc88075149e18c31"}
Nov 04 01:34:25.451331 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:25.451325    2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:34:25.451508 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:25.451491    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:25.466750 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:25.466726    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:25.633778 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:25.633685    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv" podStartSLOduration=8.392514822 podStartE2EDuration="23.633672391s" totalImagesPullingTime="15.241157569s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:25.633481922 +0000 UTC m=+23.928089484" watchObservedRunningTime="2026-11-04 01:34:25.633672391 +0000 UTC m=+23.928279953"
Nov 04 01:34:26.453727 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:26.453692    2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:34:26.455288 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:26.455267    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hjqq9"]
Nov 04 01:34:26.455379 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:26.455357    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:26.455463 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:26.455425    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:26.458442 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:26.458418    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-q7q6c"]
Nov 04 01:34:26.458578 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:26.458548    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:26.458671 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:26.458651    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:26.459046 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:26.459025    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hbs2c"]
Nov 04 01:34:26.459118 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:26.459101    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:26.459232 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:26.459170    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:27.455823 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:27.455795    2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:34:28.250653 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:28.250612    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:28.250814 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:28.250612    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:28.250814 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:28.250758    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:28.250814 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:28.250626    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:28.250814 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:28.250798    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:28.251021 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:28.250863    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:29.393594 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:29.393569    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zpmgq_aa398d49-e1ee-41df-8785-b765e1f35b18/dns-node-resolver/0.log"
Nov 04 01:34:30.218582 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:30.218514    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z992q_20296e8e-e25f-40e1-9b2b-4971b7e69141/node-ca/0.log"
Nov 04 01:34:30.253711 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:30.253687    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:30.253856 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:30.253762    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:30.253856 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:30.253850    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:30.253936 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:30.253864    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:30.253936 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:30.253912    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:30.254032 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:30.253993    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:30.288372 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:30.288346    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:30.288605 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:30.288590    2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:34:30.299484 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:30.299464    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:34:32.252148 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:32.251974    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:32.252641 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:32.252043    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:32.252641 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:32.252214    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:32.252641 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:32.252069    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:32.252641 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:32.252284    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:32.252641 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:32.252347    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:34.253231 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:34.253203    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:34.253670 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:34.253297    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:34.253670 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:34.253304    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:34.253670 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:34.253421    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:34.253670 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:34.253483    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:34.253670 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:34.253559    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:35.152140 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:35.152102    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs\") pod \"network-metrics-daemon-q7q6c\" (UID: \"e62bee01-e717-4484-bb3d-ff4af522a355\") " pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:35.152358 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:35.152186    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:35.152358 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:35.152254    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:34:35.152358 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:35.152287    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 01:34:35.152358 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:35.152318    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs podName:e62bee01-e717-4484-bb3d-ff4af522a355 nodeName:}" failed. No retries permitted until 2026-11-04 01:35:07.152302365 +0000 UTC m=+65.446909922 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs") pod "network-metrics-daemon-q7q6c" (UID: "e62bee01-e717-4484-bb3d-ff4af522a355") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 01:34:35.152358 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:35.152339    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret podName:134fc8ea-90c5-4b4c-86a3-7b83d950f3e5 nodeName:}" failed. No retries permitted until 2026-11-04 01:35:07.152324108 +0000 UTC m=+65.446931648 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret") pod "global-pull-secret-syncer-hjqq9" (UID: "134fc8ea-90c5-4b4c-86a3-7b83d950f3e5") : object "kube-system"/"original-pull-secret" not registered
Nov 04 01:34:35.252937 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:35.252895    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c8rj\" (UniqueName: \"kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj\") pod \"network-check-target-hbs2c\" (UID: \"ca08235e-943c-4d8d-939d-4ceaedd9cf92\") " pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:35.253103 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:35.253058    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 01:34:35.253103 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:35.253077    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 01:34:35.253103 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:35.253089    2575 projected.go:196] Error preparing data for projected volume kube-api-access-7c8rj for pod openshift-network-diagnostics/network-check-target-hbs2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:34:35.253216 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:35.253141    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj podName:ca08235e-943c-4d8d-939d-4ceaedd9cf92 nodeName:}" failed. No retries permitted until 2026-11-04 01:35:07.253126818 +0000 UTC m=+65.547734374 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7c8rj" (UniqueName: "kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj") pod "network-check-target-hbs2c" (UID: "ca08235e-943c-4d8d-939d-4ceaedd9cf92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 01:34:36.250566 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:36.250522    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:36.250566 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:36.250550    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:36.251101 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:36.250550    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:36.251101 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:36.250647    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:36.251101 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:36.250706    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:36.251101 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:36.250777    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:38.251325 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:38.251292    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:38.251325 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:38.251320    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:38.251794 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:38.251458    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:38.251794 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:38.251473    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:38.251794 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:38.251556    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:38.251794 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:38.251641    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:40.250526 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:40.250487    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:40.250923 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:40.250487    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:40.250923 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:40.250618    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:40.250923 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:40.250659    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:40.250923 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:40.250500    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:40.250923 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:40.250749    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:42.251905 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:42.251871    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:42.252515 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:42.251999    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:42.252515 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:42.252026    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:42.252515 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:42.252069    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:42.252515 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:42.252095    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:42.252515 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:42.252154    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:44.250510 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:44.250459    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:44.250997 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:44.250460    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:44.250997 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:44.250586    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:44.250997 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:44.250480    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:44.250997 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:44.250657    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:44.250997 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:44.250738    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:46.251036 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:46.250995    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:46.251036 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:46.251015    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:46.251608 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:46.251169    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:46.251608 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:46.251206    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:46.251608 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:46.251303    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:46.251608 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:46.251379    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:48.250572 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:48.250537    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:48.250572 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:48.250560    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:48.251001 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:48.250537    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:48.251001 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:48.250644    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:48.251001 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:48.250704    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:48.251001 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:48.250768    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:50.251255 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.251217    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:50.251255 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.251217    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:50.251717 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:50.251402    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7q6c" podUID="e62bee01-e717-4484-bb3d-ff4af522a355"
Nov 04 01:34:50.251717 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.251439    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:50.251717 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:50.251539    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hbs2c" podUID="ca08235e-943c-4d8d-939d-4ceaedd9cf92"
Nov 04 01:34:50.251717 ip-10-0-129-138 kubenswrapper[2575]: E1104 01:34:50.251608    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hjqq9" podUID="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5"
Nov 04 01:34:50.494745 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.494717    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeReady"
Nov 04 01:34:50.494919 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.494855    2575 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 04 01:34:50.595587 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.595548    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-76b68"]
Nov 04 01:34:50.597407 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.597383    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-76b68"
Nov 04 01:34:50.598335 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.598317    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7lrcs"]
Nov 04 01:34:50.599812 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.599796    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7lrcs"
Nov 04 01:34:50.616395 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.616369    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8zjpf"]
Nov 04 01:34:50.617934 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.617920    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:34:50.652888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.652859    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xkhwh\""
Nov 04 01:34:50.652888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.652860    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 01:34:50.653867 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.653850    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-config-volume\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:34:50.653908 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.653876    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-metrics-tls\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:34:50.653952 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.653901    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fjld\" (UniqueName: \"kubernetes.io/projected/1e5c731f-7d75-42d6-8ebd-ee742632fa3d-kube-api-access-5fjld\") pod \"ingress-canary-7lrcs\" (UID: \"1e5c731f-7d75-42d6-8ebd-ee742632fa3d\") " pod="openshift-ingress-canary/ingress-canary-7lrcs"
Nov 04 01:34:50.653990 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.653953    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-tmp-dir\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:34:50.654068 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.654053    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e5c731f-7d75-42d6-8ebd-ee742632fa3d-cert\") pod \"ingress-canary-7lrcs\" (UID: \"1e5c731f-7d75-42d6-8ebd-ee742632fa3d\") " pod="openshift-ingress-canary/ingress-canary-7lrcs"
Nov 04 01:34:50.654102 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.654091    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fqmn\" (UniqueName: \"kubernetes.io/projected/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-kube-api-access-4fqmn\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:34:50.657775 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.657750    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-76b68"]
Nov 04 01:34:50.660464 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.660423    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7lrcs"]
Nov 04 01:34:50.667062 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.667040    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 01:34:50.667194 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.667157    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 01:34:50.667194 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.667172    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 01:34:50.672619 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.672601    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 01:34:50.672675 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.672638    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 01:34:50.672675 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.672638    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 01:34:50.722247 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.722218    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8zjpf"]
Nov 04 01:34:50.754759 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.754668    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lls22\" (UniqueName: \"kubernetes.io/projected/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-kube-api-access-lls22\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:34:50.754759 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.754706    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:34:50.754969 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.754801    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-crio-socket\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:34:50.754969 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.754836    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e5c731f-7d75-42d6-8ebd-ee742632fa3d-cert\") pod \"ingress-canary-7lrcs\" (UID: \"1e5c731f-7d75-42d6-8ebd-ee742632fa3d\") " pod="openshift-ingress-canary/ingress-canary-7lrcs"
Nov 04 01:34:50.754969 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.754873    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fqmn\" (UniqueName: \"kubernetes.io/projected/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-kube-api-access-4fqmn\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:34:50.754969 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.754904    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-config-volume\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:34:50.754969 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.754925    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-metrics-tls\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:34:50.754969 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.754948    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fjld\" (UniqueName: \"kubernetes.io/projected/1e5c731f-7d75-42d6-8ebd-ee742632fa3d-kube-api-access-5fjld\") pod \"ingress-canary-7lrcs\" (UID: \"1e5c731f-7d75-42d6-8ebd-ee742632fa3d\") " pod="openshift-ingress-canary/ingress-canary-7lrcs"
Nov 04 01:34:50.755212 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.754972    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-tmp-dir\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:34:50.755212 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.754992    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-data-volume\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:34:50.755212 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.755014    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:34:50.755403 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.755382    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-tmp-dir\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:34:50.755526 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.755511    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-config-volume\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:34:50.757624 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.757605    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 01:34:50.758237 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.758174    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-gnb5c\""
Nov 04 01:34:50.758237 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.758190    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 01:34:50.758427 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.758406    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j6ln8\""
Nov 04 01:34:50.761377 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.761351    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-metrics-tls\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:34:50.767327 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.767303    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e5c731f-7d75-42d6-8ebd-ee742632fa3d-cert\") pod \"ingress-canary-7lrcs\" (UID: \"1e5c731f-7d75-42d6-8ebd-ee742632fa3d\") " pod="openshift-ingress-canary/ingress-canary-7lrcs"
Nov 04 01:34:50.817893 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.817864    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fqmn\" (UniqueName: \"kubernetes.io/projected/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-kube-api-access-4fqmn\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:34:50.818107 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.818089    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fjld\" (UniqueName: \"kubernetes.io/projected/1e5c731f-7d75-42d6-8ebd-ee742632fa3d-kube-api-access-5fjld\") pod \"ingress-canary-7lrcs\" (UID: \"1e5c731f-7d75-42d6-8ebd-ee742632fa3d\") " pod="openshift-ingress-canary/ingress-canary-7lrcs"
Nov 04 01:34:50.855667 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.855627    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-data-volume\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:34:50.855667 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.855662    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:34:50.855921 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.855682    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lls22\" (UniqueName: \"kubernetes.io/projected/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-kube-api-access-lls22\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:34:50.855921 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.855737    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:34:50.855921 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.855796    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-crio-socket\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:34:50.856072 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.855942    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-crio-socket\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:34:50.856162 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.856141    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-data-volume\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:34:50.856342 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.856321    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:34:50.858080 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.858065    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:34:50.909490 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.909432    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-76b68"
Nov 04 01:34:50.912875 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.912858    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7lrcs"
Nov 04 01:34:50.920058 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.920031    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lls22\" (UniqueName: \"kubernetes.io/projected/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-kube-api-access-lls22\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:34:50.928938 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:50.928911    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:34:51.358566 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:51.358508    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7lrcs"]
Nov 04 01:34:51.361027 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:51.360998    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-76b68"]
Nov 04 01:34:51.363815 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:51.363790    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4044e7b0_63ac_4b5f_af3b_5845b75f7c6f.slice/crio-e7daeb00d50a5744358d82d076f8b8225059b60a1f030e4c13e7672a968948fa WatchSource:0}: Error finding container e7daeb00d50a5744358d82d076f8b8225059b60a1f030e4c13e7672a968948fa: Status 404 returned error can't find the container with id e7daeb00d50a5744358d82d076f8b8225059b60a1f030e4c13e7672a968948fa
Nov 04 01:34:51.409356 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:51.409335    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8zjpf"]
Nov 04 01:34:51.411735 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:51.411709    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf2ea8fb_fd39_40ee_8ad5_c2bd3db434ce.slice/crio-86205162f356bdf692e2483138eb9e186d5d9f994d51544b807266900e53264f WatchSource:0}: Error finding container 86205162f356bdf692e2483138eb9e186d5d9f994d51544b807266900e53264f: Status 404 returned error can't find the container with id 86205162f356bdf692e2483138eb9e186d5d9f994d51544b807266900e53264f
Nov 04 01:34:51.508946 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:51.508908    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8zjpf" event={"ID":"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce","Type":"ContainerStarted","Data":"48bf39859354ae422d213dd4472241c119fa5b56b56f3c089e2999bba49b580b"}
Nov 04 01:34:51.509108 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:51.508969    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8zjpf" event={"ID":"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce","Type":"ContainerStarted","Data":"86205162f356bdf692e2483138eb9e186d5d9f994d51544b807266900e53264f"}
Nov 04 01:34:51.510007 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:51.509985    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7lrcs" event={"ID":"1e5c731f-7d75-42d6-8ebd-ee742632fa3d","Type":"ContainerStarted","Data":"1ff7d6b1c20e0f6b1137ac080e30bd0af5e1da81a648bf34794afb2c8ec46bc1"}
Nov 04 01:34:51.511009 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:51.510991    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-76b68" event={"ID":"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f","Type":"ContainerStarted","Data":"e7daeb00d50a5744358d82d076f8b8225059b60a1f030e4c13e7672a968948fa"}
Nov 04 01:34:52.255275 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:52.255242    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:34:52.255465 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:52.255364    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:34:52.255912 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:52.255637    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:34:52.397204 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:52.397177    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 01:34:52.423905 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:52.423871    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mh5l6\""
Nov 04 01:34:52.424071 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:52.423967    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 01:34:52.424071 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:52.424025    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-vqsf7\""
Nov 04 01:34:52.424171 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:52.424106    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 01:34:52.424171 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:52.424154    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 01:34:52.424260 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:52.424190    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-ldpmc\""
Nov 04 01:34:53.520884 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:53.520752    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8zjpf" event={"ID":"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce","Type":"ContainerStarted","Data":"54d485a3e3803982469d73b5c7b6945c6d1d263a13443afe260e232ccb9d5a47"}
Nov 04 01:34:53.747348 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:53.747248    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zpmgq_aa398d49-e1ee-41df-8785-b765e1f35b18/dns-node-resolver/0.log"
Nov 04 01:34:53.995809 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:53.995777    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"]
Nov 04 01:34:53.997993 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:53.997940    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:34:54.052078 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.052038    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 01:34:54.052324 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.052308    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 01:34:54.052442 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.052426    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Nov 04 01:34:54.052619 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.052606    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-9mnjb\""
Nov 04 01:34:54.052687 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.052667    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Nov 04 01:34:54.052687 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.052665    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 04 01:34:54.057379 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.057351    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"]
Nov 04 01:34:54.086541 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.086504    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:34:54.086726 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.086566    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:34:54.086726 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.086682    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq7r6\" (UniqueName: \"kubernetes.io/projected/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-kube-api-access-dq7r6\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:34:54.086842 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.086786    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:34:54.187540 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.187504    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:34:54.187734 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.187567    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:34:54.187734 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.187603    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:34:54.187734 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.187636    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq7r6\" (UniqueName: \"kubernetes.io/projected/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-kube-api-access-dq7r6\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:34:54.188397 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.188371    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:34:54.197483 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.190540    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:34:54.197483 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.190572    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:34:54.251113 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.251043    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq7r6\" (UniqueName: \"kubernetes.io/projected/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-kube-api-access-dq7r6\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:34:54.311137 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.311097    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:34:54.526373 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.526291    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-76b68" event={"ID":"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f","Type":"ContainerStarted","Data":"b5d3c9a8623697afc1af5f2f07abb06ac2ee32f6c6aec76815421bb90f82c985"}
Nov 04 01:34:54.526373 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.526354    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-76b68" event={"ID":"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f","Type":"ContainerStarted","Data":"4e72f43ad1930386001f48442f30965648c34d2246e4e1d84a2dd44e67f82636"}
Nov 04 01:34:54.526925 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.526488    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-76b68"
Nov 04 01:34:54.648650 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.648604    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-76b68" podStartSLOduration=2.542219374 podStartE2EDuration="4.648590864s" totalImagesPullingTime="2.10637149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:34:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:54.648178802 +0000 UTC m=+52.942786363" watchObservedRunningTime="2026-11-04 01:34:54.648590864 +0000 UTC m=+52.943198426"
Nov 04 01:34:54.688278 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.688249    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"]
Nov 04 01:34:54.727984 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:34:54.727956    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb15eb62_434d_4cdd_a2dd_d1e069bb6abb.slice/crio-35ddf6e87039025375b3fb74fa96171ce2e40a076548a59380f15444aef36a7f WatchSource:0}: Error finding container 35ddf6e87039025375b3fb74fa96171ce2e40a076548a59380f15444aef36a7f: Status 404 returned error can't find the container with id 35ddf6e87039025375b3fb74fa96171ce2e40a076548a59380f15444aef36a7f
Nov 04 01:34:54.951842 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:54.951809    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z992q_20296e8e-e25f-40e1-9b2b-4971b7e69141/node-ca/0.log"
Nov 04 01:34:55.530888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:55.530848    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8zjpf" event={"ID":"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce","Type":"ContainerStarted","Data":"ffe344ca280dd08196958b0ada676f25a7111b94706e000cc37c6fa8e0a73ca0"}
Nov 04 01:34:55.532060 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:55.532035    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd" event={"ID":"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb","Type":"ContainerStarted","Data":"35ddf6e87039025375b3fb74fa96171ce2e40a076548a59380f15444aef36a7f"}
Nov 04 01:34:55.533496 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:55.533467    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7lrcs" event={"ID":"1e5c731f-7d75-42d6-8ebd-ee742632fa3d","Type":"ContainerStarted","Data":"9e22db4c9e88aa5933f2286af9c9fd8177f5010e527288bf126a9c3bf10c11bd"}
Nov 04 01:34:55.652289 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:55.652237    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8zjpf" podStartSLOduration=2.631825031 podStartE2EDuration="5.652224659s" totalImagesPullingTime="3.020399628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:34:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:55.651892289 +0000 UTC m=+53.946499851" watchObservedRunningTime="2026-11-04 01:34:55.652224659 +0000 UTC m=+53.946832212"
Nov 04 01:34:55.770280 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:55.770227    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7lrcs" podStartSLOduration=2.5711778560000003 podStartE2EDuration="5.77021313s" totalImagesPullingTime="3.199035274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:34:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:55.769634578 +0000 UTC m=+54.064242137" watchObservedRunningTime="2026-11-04 01:34:55.77021313 +0000 UTC m=+54.064820692"
Nov 04 01:34:56.539020 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:56.538980    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd" event={"ID":"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb","Type":"ContainerStarted","Data":"87a03b9873bc8a43894dd9a4d04f4f07d55b48f96b3787d3658298dfe34d4219"}
Nov 04 01:34:56.539487 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:56.539039    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd" event={"ID":"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb","Type":"ContainerStarted","Data":"711d91f2281c0d1026a2027cd3af8babc93ff2c1664ff7570fd01dae3fad14bd"}
Nov 04 01:34:56.688145 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:56.688096    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd" podStartSLOduration=2.071895631 podStartE2EDuration="3.688082792s" totalImagesPullingTime="1.616187161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:34:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:56.687649222 +0000 UTC m=+54.982256785" watchObservedRunningTime="2026-11-04 01:34:56.688082792 +0000 UTC m=+54.982690352"
Nov 04 01:34:59.614558 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.614523    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"]
Nov 04 01:34:59.616545 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.616530    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.665813 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.665779    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-27c6h"]
Nov 04 01:34:59.667705 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.667689    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.685598 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.685574    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Nov 04 01:34:59.716271 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.716236    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-nbgcc\""
Nov 04 01:34:59.716410 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.716292    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Nov 04 01:34:59.721549 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.721525    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Nov 04 01:34:59.721657 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.721628    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 04 01:34:59.722515 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.722499    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"]
Nov 04 01:34:59.732787 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.732761    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/469986a0-2b14-47a4-8f4d-3a1456447cdb-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.732829 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.732792    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/469986a0-2b14-47a4-8f4d-3a1456447cdb-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.732829 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.732819    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.732976 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.732958    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.733047 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.733030    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nrqq\" (UniqueName: \"kubernetes.io/projected/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-api-access-4nrqq\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.733090 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.733068    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.770581 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.770540    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 04 01:34:59.770581 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.770549    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 04 01:34:59.770581 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.770585    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bnxvr\""
Nov 04 01:34:59.834403 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.834361    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.834580 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.834410    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.834580 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.834436    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nrqq\" (UniqueName: \"kubernetes.io/projected/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-api-access-4nrqq\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.834580 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.834536    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-wtmp\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.834580 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.834574    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrqc6\" (UniqueName: \"kubernetes.io/projected/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-kube-api-access-lrqc6\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.834739 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.834607    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/469986a0-2b14-47a4-8f4d-3a1456447cdb-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.834739 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.834632    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/469986a0-2b14-47a4-8f4d-3a1456447cdb-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.834739 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.834682    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.834739 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.834718    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-accelerators-collector-config\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.834892 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.834754    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-metrics-client-ca\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.834892 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.834780    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-textfile\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.834962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.834910    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-tls\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.835007 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.834968    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.835007 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.834984    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/469986a0-2b14-47a4-8f4d-3a1456447cdb-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.835095 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.835036    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-sys\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.835095 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.835067    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-root\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.835569 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.835546    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.835656 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.835568    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/469986a0-2b14-47a4-8f4d-3a1456447cdb-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.837114 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.837089    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.837192 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.837177    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.899669 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.899600    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nrqq\" (UniqueName: \"kubernetes.io/projected/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-api-access-4nrqq\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.928433 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.928400    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:34:59.935774 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.935750    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-sys\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.935879 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.935786    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-root\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.935879 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.935806    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.936011 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.935875    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-sys\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.936011 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.935891    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-root\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.936011 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.935937    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-wtmp\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.936011 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.935974    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrqc6\" (UniqueName: \"kubernetes.io/projected/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-kube-api-access-lrqc6\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.936011 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.936009    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-accelerators-collector-config\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.936228 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.936038    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-metrics-client-ca\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.936228 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.936061    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-textfile\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.936228 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.936097    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-tls\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.936228 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.936104    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-wtmp\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.936624 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.936602    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-textfile\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.936870 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.936668    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-metrics-client-ca\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.937024 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.937000    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-accelerators-collector-config\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.938409 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.938375    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-tls\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.938521 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.938487    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:34:59.998144 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:34:59.998117    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrqc6\" (UniqueName: \"kubernetes.io/projected/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-kube-api-access-lrqc6\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:00.279181 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:00.279147    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:00.288810 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:35:00.288781    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ecc4bea_722e_4b62_a5f9_76cc8e5d9a4f.slice/crio-7e89eb172b9b14a9b75ad94924e7d50441b3927e6200f523840e9345b97d205e WatchSource:0}: Error finding container 7e89eb172b9b14a9b75ad94924e7d50441b3927e6200f523840e9345b97d205e: Status 404 returned error can't find the container with id 7e89eb172b9b14a9b75ad94924e7d50441b3927e6200f523840e9345b97d205e
Nov 04 01:35:00.345905 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:00.345875    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"]
Nov 04 01:35:00.348683 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:35:00.348649    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod469986a0_2b14_47a4_8f4d_3a1456447cdb.slice/crio-40959d56f16d23772a65b94dc695188164936b718ccfb8aaca0441818f2fb587 WatchSource:0}: Error finding container 40959d56f16d23772a65b94dc695188164936b718ccfb8aaca0441818f2fb587: Status 404 returned error can't find the container with id 40959d56f16d23772a65b94dc695188164936b718ccfb8aaca0441818f2fb587
Nov 04 01:35:00.558516 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:00.558409    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch" event={"ID":"469986a0-2b14-47a4-8f4d-3a1456447cdb","Type":"ContainerStarted","Data":"40959d56f16d23772a65b94dc695188164936b718ccfb8aaca0441818f2fb587"}
Nov 04 01:35:00.559541 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:00.559518    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27c6h" event={"ID":"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f","Type":"ContainerStarted","Data":"7e89eb172b9b14a9b75ad94924e7d50441b3927e6200f523840e9345b97d205e"}
Nov 04 01:35:02.569120 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:02.569079    2575 generic.go:356] "Generic (PLEG): container finished" podID="4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f" containerID="0f0be9a1950d9679b2d729820067d1de5dc56227a3e97856f86727dc22f92372" exitCode=0
Nov 04 01:35:02.569649 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:02.569187    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27c6h" event={"ID":"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f","Type":"ContainerDied","Data":"0f0be9a1950d9679b2d729820067d1de5dc56227a3e97856f86727dc22f92372"}
Nov 04 01:35:02.571764 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:02.571738    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch" event={"ID":"469986a0-2b14-47a4-8f4d-3a1456447cdb","Type":"ContainerStarted","Data":"c6230efdacf84d0ad336c66ec79d18ff5bef4dddb1449a7f9fca7188e7f0623f"}
Nov 04 01:35:02.571998 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:02.571798    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch" event={"ID":"469986a0-2b14-47a4-8f4d-3a1456447cdb","Type":"ContainerStarted","Data":"fe4fd5fa53877e867e89700c23a278eef8538628ddecc19371a680cea01fbae8"}
Nov 04 01:35:02.571998 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:02.571827    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch" event={"ID":"469986a0-2b14-47a4-8f4d-3a1456447cdb","Type":"ContainerStarted","Data":"e69e29a442b3b774815ed078c717dad0e414d22460849c7d76ce713770927bb3"}
Nov 04 01:35:02.866728 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:02.866632    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch" podStartSLOduration=2.233955322 podStartE2EDuration="3.866617658s" totalImagesPullingTime="1.632662336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:34:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:35:02.865858529 +0000 UTC m=+61.160466094" watchObservedRunningTime="2026-11-04 01:35:02.866617658 +0000 UTC m=+61.161225241"
Nov 04 01:35:03.576794 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:03.576747    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27c6h" event={"ID":"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f","Type":"ContainerStarted","Data":"374ebfe1b1e82f9ae63f07a4a0439f695d285b51cfbf7b7215570569bcc6b1f4"}
Nov 04 01:35:03.577189 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:03.576848    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27c6h" event={"ID":"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f","Type":"ContainerStarted","Data":"b494ee6be235ca356e9d3c44d8a8ce7efa57ceba9197026bd70c74f5fd5ee23c"}
Nov 04 01:35:03.699284 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:03.699225    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-27c6h" podStartSLOduration=3.309163841 podStartE2EDuration="4.699211176s" totalImagesPullingTime="1.310291096s" totalInitContainerRuntime="79.756239ms" isStatefulPod=true podCreationTimestamp="2026-11-04 01:34:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:35:03.698465265 +0000 UTC m=+61.993072817" watchObservedRunningTime="2026-11-04 01:35:03.699211176 +0000 UTC m=+61.993818781"
Nov 04 01:35:04.366903 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:04.366868    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"]
Nov 04 01:35:04.368771 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:04.368756    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"
Nov 04 01:35:04.426851 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:04.426821    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-g7lb4\""
Nov 04 01:35:04.427220 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:04.427192    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Nov 04 01:35:04.428270 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:04.428239    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"]
Nov 04 01:35:04.480132 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:04.480100    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-w4mgl\" (UID: \"0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"
Nov 04 01:35:04.480308 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:04.480207    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28b8j\" (UniqueName: \"kubernetes.io/projected/0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc-kube-api-access-28b8j\") pod \"monitoring-plugin-77f4f76594-w4mgl\" (UID: \"0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"
Nov 04 01:35:04.535677 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:04.535650    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-76b68"
Nov 04 01:35:04.580509 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:04.580485    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28b8j\" (UniqueName: \"kubernetes.io/projected/0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc-kube-api-access-28b8j\") pod \"monitoring-plugin-77f4f76594-w4mgl\" (UID: \"0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"
Nov 04 01:35:04.580887 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:04.580532    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-w4mgl\" (UID: \"0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"
Nov 04 01:35:04.583016 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:04.582995    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-w4mgl\" (UID: \"0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"
Nov 04 01:35:04.643091 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:04.643017    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28b8j\" (UniqueName: \"kubernetes.io/projected/0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc-kube-api-access-28b8j\") pod \"monitoring-plugin-77f4f76594-w4mgl\" (UID: \"0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"
Nov 04 01:35:04.678439 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:04.678409    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"
Nov 04 01:35:05.050180 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.050155    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"]
Nov 04 01:35:05.052732 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:35:05.052708    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b80cbf4_91a6_4716_ace2_9fd6d52ac7bc.slice/crio-d96830bd110f9ffe63a8f5c97af594eba61bab50ce0382a4e784b76c72dcc8bd WatchSource:0}: Error finding container d96830bd110f9ffe63a8f5c97af594eba61bab50ce0382a4e784b76c72dcc8bd: Status 404 returned error can't find the container with id d96830bd110f9ffe63a8f5c97af594eba61bab50ce0382a4e784b76c72dcc8bd
Nov 04 01:35:05.583952 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.583923    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl" event={"ID":"0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc","Type":"ContainerStarted","Data":"d96830bd110f9ffe63a8f5c97af594eba61bab50ce0382a4e784b76c72dcc8bd"}
Nov 04 01:35:05.678076 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.678044    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cdfff847d-84rl8"]
Nov 04 01:35:05.680723 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.680702    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.735354 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.735325    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 04 01:35:05.785902 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.785864    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 04 01:35:05.785902 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.785865    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 04 01:35:05.786147 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.786081    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 04 01:35:05.786147 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.786092    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 04 01:35:05.791374 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.791342    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-oauth-config\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.791780 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.791762    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-serving-cert\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.791847 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.791790    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nrq5\" (UniqueName: \"kubernetes.io/projected/010e4cd1-587e-43a7-9f58-93ca75a074df-kube-api-access-4nrq5\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.791847 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.791811    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-oauth-serving-cert\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.791946 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.791892    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-console-config\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.792014 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.791997    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-trusted-ca-bundle\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.792069 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.792053    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/010e4cd1-587e-43a7-9f58-93ca75a074df-tmp\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.792133 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.792118    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-service-ca\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.889735 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.889649    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Nov 04 01:35:05.889735 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.889715    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 04 01:35:05.889949 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.889764    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2jjdl\""
Nov 04 01:35:05.889949 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.889849    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 04 01:35:05.890700 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.890686    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cdfff847d-84rl8"]
Nov 04 01:35:05.893201 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.893173    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-trusted-ca-bundle\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.893269 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.893224    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/010e4cd1-587e-43a7-9f58-93ca75a074df-tmp\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.893312 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.893270    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-service-ca\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.893363 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.893332    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-oauth-config\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.893363 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.893350    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-serving-cert\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.893430 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.893372    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nrq5\" (UniqueName: \"kubernetes.io/projected/010e4cd1-587e-43a7-9f58-93ca75a074df-kube-api-access-4nrq5\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.893430 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.893398    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-oauth-serving-cert\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.893539 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.893431    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-console-config\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.894072 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.894046    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/010e4cd1-587e-43a7-9f58-93ca75a074df-tmp\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.894333 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.894131    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-oauth-serving-cert\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.894333 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.894139    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-console-config\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.894333 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.894156    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-service-ca\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.894333 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.894140    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-trusted-ca-bundle\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.896659 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.896630    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-oauth-config\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.896754 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.896714    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-serving-cert\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.956305 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.956267    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nrq5\" (UniqueName: \"kubernetes.io/projected/010e4cd1-587e-43a7-9f58-93ca75a074df-kube-api-access-4nrq5\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:05.991498 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:05.991441    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:06.373922 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:06.373891    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cdfff847d-84rl8"]
Nov 04 01:35:06.404716 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:35:06.404677    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod010e4cd1_587e_43a7_9f58_93ca75a074df.slice/crio-46bc5e27140f7feda0532059508a8f407c36ea862d2005918b83dd88f2b0c7f6 WatchSource:0}: Error finding container 46bc5e27140f7feda0532059508a8f407c36ea862d2005918b83dd88f2b0c7f6: Status 404 returned error can't find the container with id 46bc5e27140f7feda0532059508a8f407c36ea862d2005918b83dd88f2b0c7f6
Nov 04 01:35:06.589770 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:06.589731    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cdfff847d-84rl8" event={"ID":"010e4cd1-587e-43a7-9f58-93ca75a074df","Type":"ContainerStarted","Data":"46bc5e27140f7feda0532059508a8f407c36ea862d2005918b83dd88f2b0c7f6"}
Nov 04 01:35:07.206161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.206131    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:35:07.206161 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.206195    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs\") pod \"network-metrics-daemon-q7q6c\" (UID: \"e62bee01-e717-4484-bb3d-ff4af522a355\") " pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:35:07.259395 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.259363    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 01:35:07.259395 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.259363    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 01:35:07.268740 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.268716    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:35:07.268740 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.268734    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs\") pod \"network-metrics-daemon-q7q6c\" (UID: \"e62bee01-e717-4484-bb3d-ff4af522a355\") " pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:35:07.306694 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.306663    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c8rj\" (UniqueName: \"kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj\") pod \"network-check-target-hbs2c\" (UID: \"ca08235e-943c-4d8d-939d-4ceaedd9cf92\") " pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:35:07.333239 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.333212    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mh5l6\""
Nov 04 01:35:07.338391 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.338373    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:35:07.342221 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.342205    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-ldpmc\""
Nov 04 01:35:07.349515 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.349492    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:35:07.359734 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.359676    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 01:35:07.420516 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.420483    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 01:35:07.431688 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.431632    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c8rj\" (UniqueName: \"kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj\") pod \"network-check-target-hbs2c\" (UID: \"ca08235e-943c-4d8d-939d-4ceaedd9cf92\") " pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:35:07.595389 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.595297    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl" event={"ID":"0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc","Type":"ContainerStarted","Data":"1dc59c8e6f0b550633e00e378ce2278c0f5b841df1dd1743e5f99980dcfe553a"}
Nov 04 01:35:07.595820 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.595570    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"
Nov 04 01:35:07.600896 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.600871    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"
Nov 04 01:35:07.637587 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.637536    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-vqsf7\""
Nov 04 01:35:07.644632 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.644604    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:35:07.720296 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.720221    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl" podStartSLOduration=2.064320424 podStartE2EDuration="3.720202494s" totalImagesPullingTime="1.65588207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:35:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:35:07.717121465 +0000 UTC m=+66.011729029" watchObservedRunningTime="2026-11-04 01:35:07.720202494 +0000 UTC m=+66.014810057"
Nov 04 01:35:07.720601 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.720573    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-q7q6c"]
Nov 04 01:35:07.757067 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:35:07.757032    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode62bee01_e717_4484_bb3d_ff4af522a355.slice/crio-37d0f2edd1924684ff91503d18feac64b6b4ff2bc8599be69f80de3c41c9fccc WatchSource:0}: Error finding container 37d0f2edd1924684ff91503d18feac64b6b4ff2bc8599be69f80de3c41c9fccc: Status 404 returned error can't find the container with id 37d0f2edd1924684ff91503d18feac64b6b4ff2bc8599be69f80de3c41c9fccc
Nov 04 01:35:07.769291 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:07.769262    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hjqq9"]
Nov 04 01:35:07.771824 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:35:07.771796    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod134fc8ea_90c5_4b4c_86a3_7b83d950f3e5.slice/crio-b4fb007454368a64771fd1e72d20bcd0cb130ed0877b8d84d9f479d4f4f3b0d8 WatchSource:0}: Error finding container b4fb007454368a64771fd1e72d20bcd0cb130ed0877b8d84d9f479d4f4f3b0d8: Status 404 returned error can't find the container with id b4fb007454368a64771fd1e72d20bcd0cb130ed0877b8d84d9f479d4f4f3b0d8
Nov 04 01:35:08.032723 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.032702    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hbs2c"]
Nov 04 01:35:08.034639 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:35:08.034612    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca08235e_943c_4d8d_939d_4ceaedd9cf92.slice/crio-8a44dae6456170696ad7634257802f637ee6dbc19006508e859beaca7e039bb9 WatchSource:0}: Error finding container 8a44dae6456170696ad7634257802f637ee6dbc19006508e859beaca7e039bb9: Status 404 returned error can't find the container with id 8a44dae6456170696ad7634257802f637ee6dbc19006508e859beaca7e039bb9
Nov 04 01:35:08.600392 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.600359    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hbs2c" event={"ID":"ca08235e-943c-4d8d-939d-4ceaedd9cf92","Type":"ContainerStarted","Data":"8a44dae6456170696ad7634257802f637ee6dbc19006508e859beaca7e039bb9"}
Nov 04 01:35:08.601603 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.601566    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hjqq9" event={"ID":"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5","Type":"ContainerStarted","Data":"b4fb007454368a64771fd1e72d20bcd0cb130ed0877b8d84d9f479d4f4f3b0d8"}
Nov 04 01:35:08.602962 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.602937    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q7q6c" event={"ID":"e62bee01-e717-4484-bb3d-ff4af522a355","Type":"ContainerStarted","Data":"37d0f2edd1924684ff91503d18feac64b6b4ff2bc8599be69f80de3c41c9fccc"}
Nov 04 01:35:08.826085 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.826049    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 01:35:08.838041 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.837626    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.896887 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.896616    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Nov 04 01:35:08.896887 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.896760    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Nov 04 01:35:08.920170 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919590    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920170 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919622    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920170 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919641    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920170 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919657    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920170 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919674    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920170 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919689    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920170 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919707    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v72sb\" (UniqueName: \"kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-kube-api-access-v72sb\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920170 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919728    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920170 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919747    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920170 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919774    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920170 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919794    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920170 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919808    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920170 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919827    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920170 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919848    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920170 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919865    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-config-out\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920170 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919884    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-config\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920777 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919897    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920777 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919912    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920777 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919928    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.920777 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.919942    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-web-config\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:08.943297 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.943232    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Nov 04 01:35:08.944074 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.944053    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Nov 04 01:35:08.946966 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.944435    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2bmnb\""
Nov 04 01:35:08.947148 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.947073    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 04 01:35:08.947148 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.947127    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Nov 04 01:35:08.947291 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.947188    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Nov 04 01:35:08.948720 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.947986    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Nov 04 01:35:08.948720 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.948581    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Nov 04 01:35:08.949541 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:08.949515    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 01:35:09.021115 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.020954    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021115 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.020992    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021115 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.021014    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021115 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.021029    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021115 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.021049    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021115 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.021071    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-config-out\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021115 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.021091    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-config\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021115 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.021106    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021115 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.021121    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021611 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.021138    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021611 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.021157    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-web-config\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021611 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.021173    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021611 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.021191    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021611 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.021207    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021611 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.021225    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021611 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.021242    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021611 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.021256    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021611 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.021277    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v72sb\" (UniqueName: \"kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-kube-api-access-v72sb\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021611 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.021304    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.021611 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.021326    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.023210 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.022927    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.023925 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.023616    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.024529 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.024083    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.024529 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.024286    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.024529 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.024292    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.024529 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.024489    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.027785 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.027749    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.027928 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.027879    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-config\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.029583 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.029063    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.029583 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.029545    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.030234 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.030215    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.031060 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.031027    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-config-out\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.047785 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.047687    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Nov 04 01:35:09.049851 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.049329    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Nov 04 01:35:09.049851 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.049383    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Nov 04 01:35:09.049851 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.049399    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Nov 04 01:35:09.049851 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.049488    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Nov 04 01:35:09.049851 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.049490    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1eqn9lagl0d1b\""
Nov 04 01:35:09.049851 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.049542    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Nov 04 01:35:09.054994 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.054779    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.057776 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.057597    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.057776 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.057723    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.058781 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.058729    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.061535 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.061512    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.061887 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.061869    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-web-config\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.063427 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.063400    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.090046 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.089968    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v72sb\" (UniqueName: \"kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-kube-api-access-v72sb\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.171888 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.171830    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:09.638808 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:09.638690    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 01:35:09.641229 ip-10-0-129-138 kubenswrapper[2575]: W1104 01:35:09.641197    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfd725f9_4be0_4be3_bb6a_898213f63761.slice/crio-355a1c2d04ae568f6664e694146974b8ab351c306ea8bb7e39207174d85a98bf WatchSource:0}: Error finding container 355a1c2d04ae568f6664e694146974b8ab351c306ea8bb7e39207174d85a98bf: Status 404 returned error can't find the container with id 355a1c2d04ae568f6664e694146974b8ab351c306ea8bb7e39207174d85a98bf
Nov 04 01:35:10.614570 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:10.614479    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cdfff847d-84rl8" event={"ID":"010e4cd1-587e-43a7-9f58-93ca75a074df","Type":"ContainerStarted","Data":"1f0b7dc8e7c491c3a0a47bf377f8f668e89df1ffd42cfe2a1eeda896853a15d1"}
Nov 04 01:35:10.617069 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:10.617038    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q7q6c" event={"ID":"e62bee01-e717-4484-bb3d-ff4af522a355","Type":"ContainerStarted","Data":"8ee527251e9bd0fcd1fd118c9d01304a2ddfa7fd15e74bba2a2d515faad6edbd"}
Nov 04 01:35:10.617213 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:10.617092    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q7q6c" event={"ID":"e62bee01-e717-4484-bb3d-ff4af522a355","Type":"ContainerStarted","Data":"88254f40f15582e2705bae6cfc4d7f26d0e32d9fd4a2e45570d27ab248c2c5a0"}
Nov 04 01:35:10.619014 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:10.618988    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd725f9-4be0-4be3-bb6a-898213f63761","Type":"ContainerStarted","Data":"355a1c2d04ae568f6664e694146974b8ab351c306ea8bb7e39207174d85a98bf"}
Nov 04 01:35:10.736916 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:10.736856    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6cdfff847d-84rl8" podStartSLOduration=1.801001168 podStartE2EDuration="5.736836877s" totalImagesPullingTime="3.935835709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:35:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:35:10.735915997 +0000 UTC m=+69.030523560" watchObservedRunningTime="2026-11-04 01:35:10.736836877 +0000 UTC m=+69.031444439"
Nov 04 01:35:10.860481 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:10.860398    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-q7q6c" podStartSLOduration=67.098747804 podStartE2EDuration="1m8.860376552s" totalImagesPullingTime="1.761628748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:35:10.859541537 +0000 UTC m=+69.154149101" watchObservedRunningTime="2026-11-04 01:35:10.860376552 +0000 UTC m=+69.154984117"
Nov 04 01:35:11.627529 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:11.627491    2575 generic.go:356] "Generic (PLEG): container finished" podID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerID="a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7" exitCode=0
Nov 04 01:35:11.627711 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:11.627577    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd725f9-4be0-4be3-bb6a-898213f63761","Type":"ContainerDied","Data":"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7"}
Nov 04 01:35:13.637671 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:13.637633    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hbs2c" event={"ID":"ca08235e-943c-4d8d-939d-4ceaedd9cf92","Type":"ContainerStarted","Data":"0604cf7f2837763426d9bf6790bbe6c0057c5186a21ff14a790be7c6e6c33238"}
Nov 04 01:35:13.638126 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:13.637834    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:35:13.755417 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:13.755354    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hbs2c" podStartSLOduration=67.081868342 podStartE2EDuration="1m11.75534009s" totalImagesPullingTime="4.673471748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:35:13.755298598 +0000 UTC m=+72.049906153" watchObservedRunningTime="2026-11-04 01:35:13.75534009 +0000 UTC m=+72.049947651"
Nov 04 01:35:14.646287 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:14.646253    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hjqq9" event={"ID":"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5","Type":"ContainerStarted","Data":"5ca7432f61c4f087f140dd81038050acec97b006bc7112e78ebea6ebd24cb381"}
Nov 04 01:35:14.764662 ip-10-0-129-138 kubenswrapper[2575]: I1104 01:35:14.764549    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hjqq9" podStartSLOduration=66.056947335 podStartE2EDuration="1m12.76453506s" totalImagesPullingTime="6.707587725s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:35:14.764388533 +0000 UTC m=+73.058996095" watchObservedRunningTime="2026-11-04 01:35:14.76453506 +0000 UTC m=+73.059142622"
Nov 04 01:35:14.806873 ip-10-0-129-138 systemd[1]: Stopping Kubernetes Kubelet...
Nov 04 01:35:14.817245 ip-10-0-129-138 systemd[1]: kubelet.service: Deactivated successfully.
Nov 04 01:35:14.817435 ip-10-0-129-138 systemd[1]: Stopped Kubernetes Kubelet.
Nov 04 01:35:14.817519 ip-10-0-129-138 systemd[1]: kubelet.service: Consumed 4.152s CPU time.
Nov 04 01:35:14.825293 ip-10-0-129-138 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 01:35:14.918131 ip-10-0-129-138 kubenswrapper[6116]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:35:14.918131 ip-10-0-129-138 kubenswrapper[6116]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 01:35:14.918131 ip-10-0-129-138 kubenswrapper[6116]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:35:14.918131 ip-10-0-129-138 kubenswrapper[6116]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 01:35:14.920604 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920514    6116 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:35:14.920604 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920601    6116 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:35:14.920604 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920607    6116 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:35:14.920604 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920611    6116 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920615    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920620    6116 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920623    6116 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920625    6116 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920628    6116 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920632    6116 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920636    6116 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920641    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920645    6116 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920649    6116 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920652    6116 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920654    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920657    6116 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920660    6116 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920662    6116 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920672    6116 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920675    6116 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920680    6116 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:35:14.920826 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920684    6116 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920688    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920692    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920695    6116 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920698    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920701    6116 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920703    6116 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920706    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920708    6116 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920712    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920717    6116 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920721    6116 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920725    6116 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920728    6116 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920730    6116 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920732    6116 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920735    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920739    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920742    6116 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920746    6116 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:35:14.921353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920750    6116 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920753    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920756    6116 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920759    6116 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920761    6116 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920764    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920766    6116 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920769    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920771    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920774    6116 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920777    6116 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920780    6116 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920783    6116 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920785    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920788    6116 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920793    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920797    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920801    6116 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920805    6116 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:35:14.921952 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920810    6116 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920813    6116 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920816    6116 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920819    6116 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920821    6116 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920824    6116 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920826    6116 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920829    6116 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920831    6116 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920834    6116 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920837    6116 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920841    6116 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920845    6116 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920847    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920850    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920852    6116 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920855    6116 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920858    6116 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920860    6116 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920863    6116 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:35:14.922582 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920865    6116 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920868    6116 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920871    6116 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920876    6116 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920880    6116 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920884    6116 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920889    6116 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920893    6116 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920896    6116 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920898    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920901    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920903    6116 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920907    6116 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920910    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920912    6116 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920915    6116 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920917    6116 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920920    6116 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920923    6116 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920925    6116 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:35:14.923188 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920928    6116 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920930    6116 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920933    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920935    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.920938    6116 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.921972    6116 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.921983    6116 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.921986    6116 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.921989    6116 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.921993    6116 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.921998    6116 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922001    6116 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922004    6116 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922007    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922012    6116 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922018    6116 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922022    6116 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922027    6116 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922031    6116 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:35:14.923846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922036    6116 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922039    6116 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922042    6116 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922044    6116 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922047    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922049    6116 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922054    6116 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922058    6116 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922062    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922065    6116 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922068    6116 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922070    6116 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922073    6116 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922075    6116 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922078    6116 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922080    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922083    6116 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922085    6116 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922092    6116 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922095    6116 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:35:14.924579 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922097    6116 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922100    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922103    6116 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922105    6116 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922108    6116 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922110    6116 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922113    6116 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922116    6116 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922120    6116 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922124    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922128    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922133    6116 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922139    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922144    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922148    6116 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922153    6116 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922157    6116 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922162    6116 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922166    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922170    6116 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:35:14.925280 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922174    6116 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922179    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922184    6116 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922188    6116 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922192    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922194    6116 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922197    6116 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922199    6116 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922202    6116 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922204    6116 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922208    6116 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922212    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922214    6116 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922217    6116 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922220    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922222    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922225    6116 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922228    6116 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922230    6116 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:35:14.925938 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922234    6116 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922237    6116 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922241    6116 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922245    6116 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922249    6116 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922252    6116 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922256    6116 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922260    6116 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922264    6116 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922268    6116 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922272    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922276    6116 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922278    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922281    6116 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922284    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922286    6116 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922290    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922294    6116 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922298    6116 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:35:14.926545 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922303    6116 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922306    6116 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922308    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922311    6116 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922315    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922317    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922321    6116 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922325    6116 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922328    6116 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922330    6116 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922333    6116 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922335    6116 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922338    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922341    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922980    6116 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922988    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922991    6116 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.922997    6116 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923003    6116 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:35:14.927079 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923007    6116 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923010    6116 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923013    6116 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923015    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923018    6116 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923021    6116 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923025    6116 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923029    6116 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923032    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923037    6116 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923040    6116 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923043    6116 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923045    6116 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923048    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923051    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923053    6116 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923056    6116 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923059    6116 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923061    6116 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:35:14.927687 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923065    6116 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923070    6116 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923072    6116 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923075    6116 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923078    6116 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923080    6116 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923083    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923085    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923088    6116 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923090    6116 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923093    6116 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923097    6116 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923102    6116 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923106    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923110    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923114    6116 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923118    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923121    6116 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923123    6116 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:35:14.928229 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923126    6116 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923128    6116 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923131    6116 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923134    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923138    6116 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923143    6116 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923148    6116 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923152    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923157    6116 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923161    6116 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923166    6116 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923171    6116 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923175    6116 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923179    6116 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923187    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923192    6116 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923198    6116 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923203    6116 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923206    6116 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:35:14.928899 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923209    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923212    6116 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923215    6116 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923218    6116 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923222    6116 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923224    6116 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923227    6116 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923230    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923232    6116 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923235    6116 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923237    6116 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923241    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923245    6116 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923249    6116 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923252    6116 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923255    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923257    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923260    6116 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923262    6116 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:35:14.929635 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923265    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923268    6116 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923272    6116 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923276    6116 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923279    6116 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923282    6116 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923284    6116 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923287    6116 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923290    6116 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923294    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923297    6116 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923299    6116 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923302    6116 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923304    6116 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923307    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923309    6116 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923312    6116 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923317    6116 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923321    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923327    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:35:14.930303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923330    6116 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923332    6116 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923334    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923337    6116 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.923339    6116 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.923486    6116 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.923496    6116 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.923504    6116 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.923508    6116 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.923512    6116 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.923515    6116 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.923520    6116 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.923525    6116 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.923529    6116 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.923534    6116 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.923540    6116 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.923545    6116 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.923548    6116 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.923551    6116 flags.go:64] FLAG: --cgroup-root=""
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.923554    6116 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.923557    6116 flags.go:64] FLAG: --client-ca-file=""
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.923560    6116 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.923691    6116 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.924922    6116 flags.go:64] FLAG: --cluster-domain=""
Nov 04 01:35:14.930983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.924932    6116 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.924939    6116 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.924947    6116 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925002    6116 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925089    6116 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925172    6116 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925177    6116 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925181    6116 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925187    6116 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925193    6116 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925196    6116 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925200    6116 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925203    6116 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925209    6116 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925212    6116 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925215    6116 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925220    6116 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925225    6116 flags.go:64] FLAG: --enable-server="true"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925229    6116 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925238    6116 flags.go:64] FLAG: --event-burst="100"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925242    6116 flags.go:64] FLAG: --event-qps="50"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925245    6116 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925248    6116 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925251    6116 flags.go:64] FLAG: --eviction-hard=""
Nov 04 01:35:14.931902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925255    6116 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925258    6116 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925261    6116 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925264    6116 flags.go:64] FLAG: --eviction-soft=""
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925267    6116 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925270    6116 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925273    6116 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925276    6116 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925279    6116 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925282    6116 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925285    6116 flags.go:64] FLAG: --feature-gates=""
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925290    6116 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925295    6116 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925298    6116 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925301    6116 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925304    6116 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925310    6116 flags.go:64] FLAG: --help="false"
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925313    6116 flags.go:64] FLAG: --hostname-override="ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925317    6116 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925320    6116 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925323    6116 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925327    6116 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925331    6116 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 01:35:14.932923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925333    6116 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925337    6116 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925339    6116 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925342    6116 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925346    6116 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925350    6116 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925354    6116 flags.go:64] FLAG: --kube-reserved=""
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925360    6116 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925364    6116 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925369    6116 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925375    6116 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925379    6116 flags.go:64] FLAG: --lock-file=""
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925384    6116 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925389    6116 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925394    6116 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925410    6116 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925415    6116 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925420    6116 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925424    6116 flags.go:64] FLAG: --logging-format="text"
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925429    6116 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925432    6116 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925437    6116 flags.go:64] FLAG: --manifest-url=""
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925440    6116 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925470    6116 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925475    6116 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 01:35:14.933728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925483    6116 flags.go:64] FLAG: --max-pods="110"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925487    6116 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925490    6116 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925492    6116 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925495    6116 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925498    6116 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925501    6116 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925504    6116 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925513    6116 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925516    6116 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925519    6116 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925523    6116 flags.go:64] FLAG: --pod-cidr=""
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925528    6116 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925532    6116 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925537    6116 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925542    6116 flags.go:64] FLAG: --port="10250"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925546    6116 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925553    6116 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0856cc5f13ab9b33f"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925559    6116 flags.go:64] FLAG: --qos-reserved=""
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925563    6116 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925569    6116 flags.go:64] FLAG: --register-node="true"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925574    6116 flags.go:64] FLAG: --register-with-taints=""
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925581    6116 flags.go:64] FLAG: --registry-burst="10"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925585    6116 flags.go:64] FLAG: --registry-qps="5"
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925589    6116 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925594    6116 flags.go:64] FLAG: --reserved-memory=""
Nov 04 01:35:14.934639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925600    6116 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925605    6116 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925609    6116 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925614    6116 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925618    6116 flags.go:64] FLAG: --runonce="false"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925623    6116 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925632    6116 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925635    6116 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925640    6116 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925644    6116 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925650    6116 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925655    6116 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925661    6116 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925666    6116 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925671    6116 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925676    6116 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925682    6116 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925688    6116 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925693    6116 flags.go:64] FLAG: --system-cgroups=""
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925697    6116 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925707    6116 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925710    6116 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925713    6116 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925721    6116 flags.go:64] FLAG: --tls-min-version=""
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925723    6116 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 01:35:14.935433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925728    6116 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925732    6116 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925737    6116 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925741    6116 flags.go:64] FLAG: --v="2"
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925747    6116 flags.go:64] FLAG: --version="false"
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925751    6116 flags.go:64] FLAG: --vmodule=""
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925756    6116 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.925759    6116 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925920    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925925    6116 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925928    6116 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925931    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925933    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925938    6116 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925941    6116 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925943    6116 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925947    6116 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925952    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925957    6116 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925962    6116 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925964    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:35:14.936260 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925967    6116 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925970    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925972    6116 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925976    6116 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925979    6116 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925981    6116 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925985    6116 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925990    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.925996    6116 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926001    6116 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926004    6116 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926007    6116 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926009    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926012    6116 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926014    6116 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926017    6116 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926020    6116 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926022    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926026    6116 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:35:14.936862 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926030    6116 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926034    6116 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926039    6116 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926042    6116 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926045    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926049    6116 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926053    6116 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926057    6116 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926060    6116 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926063    6116 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926065    6116 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926068    6116 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926070    6116 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926073    6116 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926075    6116 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926078    6116 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926080    6116 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926083    6116 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926086    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:35:14.937552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926088    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926091    6116 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926093    6116 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926097    6116 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926099    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926102    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926107    6116 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926111    6116 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926116    6116 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926120    6116 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926124    6116 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926129    6116 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926133    6116 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926137    6116 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926142    6116 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926146    6116 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926150    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926155    6116 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926161    6116 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926166    6116 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:35:14.938257 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926170    6116 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926174    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926177    6116 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926180    6116 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926182    6116 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926185    6116 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926188    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926190    6116 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926193    6116 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926196    6116 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926198    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926203    6116 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926205    6116 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926208    6116 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926211    6116 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926215    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926218    6116 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926221    6116 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926223    6116 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926226    6116 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:35:14.938782 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926228    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:35:14.939390 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926230    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:35:14.939390 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926233    6116 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:35:14.939390 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926235    6116 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:35:14.939390 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926238    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:35:14.939390 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926240    6116 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:35:14.939390 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926243    6116 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:35:14.939390 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926245    6116 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:35:14.939390 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926248    6116 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:35:14.939390 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926252    6116 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:35:14.939390 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926258    6116 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:35:14.939390 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926262    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:35:14.939390 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926267    6116 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:35:14.939390 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926272    6116 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:35:14.939390 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926276    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:35:14.939390 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.926316    6116 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:35:14.939390 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926355    6116 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926360    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926363    6116 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926367    6116 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926370    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926372    6116 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926375    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926378    6116 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926381    6116 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926384    6116 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926386    6116 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926389    6116 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926393    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926397    6116 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926400    6116 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926403    6116 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926406    6116 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926409    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926411    6116 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926414    6116 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:35:14.939996 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926417    6116 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926421    6116 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926425    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926437    6116 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926440    6116 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926442    6116 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926463    6116 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926468    6116 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926472    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926476    6116 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926482    6116 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926489    6116 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926494    6116 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926499    6116 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926503    6116 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926507    6116 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926510    6116 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926514    6116 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926519    6116 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:35:14.940677 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926521    6116 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926524    6116 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926528    6116 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926530    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926533    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926537    6116 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926540    6116 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926542    6116 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926545    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926548    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926550    6116 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926553    6116 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926556    6116 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926559    6116 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926561    6116 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926564    6116 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926566    6116 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926569    6116 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926571    6116 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926575    6116 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:35:14.941201 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926577    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926580    6116 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926582    6116 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926585    6116 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926588    6116 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926592    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926596    6116 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926601    6116 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926605    6116 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926608    6116 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926610    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926613    6116 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926617    6116 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926622    6116 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926627    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926631    6116 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926633    6116 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926637    6116 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926640    6116 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926642    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:35:14.941804 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926645    6116 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926648    6116 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926650    6116 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926653    6116 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926655    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926657    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926660    6116 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926662    6116 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926665    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926670    6116 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926674    6116 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926679    6116 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926684    6116 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926687    6116 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926689    6116 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926692    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926694    6116 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926697    6116 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926700    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:35:14.942499 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926702    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926704    6116 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926707    6116 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926709    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926712    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926714    6116 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926717    6116 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.926719    6116 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.931828    6116 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.931848    6116 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.931870    6116 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.931877    6116 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.931959    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.931965    6116 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.931969    6116 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.931973    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.931978    6116 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.931983    6116 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.931987    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:35:14.943204 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.931993    6116 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.931997    6116 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932000    6116 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932003    6116 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932005    6116 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932008    6116 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932011    6116 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932013    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932016    6116 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932019    6116 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932021    6116 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932023    6116 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932028    6116 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932035    6116 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932040    6116 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932046    6116 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932051    6116 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932056    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932060    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:35:14.943772 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932063    6116 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932065    6116 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932068    6116 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932072    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932075    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932078    6116 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932081    6116 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932086    6116 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932090    6116 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932097    6116 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932103    6116 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932106    6116 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932109    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932112    6116 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932115    6116 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932117    6116 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932120    6116 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932122    6116 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932126    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:35:14.944290 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932132    6116 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932136    6116 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932140    6116 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932146    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932151    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932155    6116 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932158    6116 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932161    6116 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932163    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932167    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932169    6116 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932172    6116 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932175    6116 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932177    6116 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932181    6116 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932186    6116 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932191    6116 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932196    6116 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932201    6116 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932206    6116 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:35:14.945016 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932211    6116 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932214    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932218    6116 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932223    6116 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932228    6116 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932232    6116 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932236    6116 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932240    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932244    6116 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932249    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932254    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932259    6116 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932264    6116 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932269    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932273    6116 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932278    6116 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932283    6116 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932287    6116 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932291    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932294    6116 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:35:14.945797 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932297    6116 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932299    6116 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932302    6116 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932305    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932309    6116 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932313    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932317    6116 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932321    6116 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932325    6116 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932327    6116 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932331    6116 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932334    6116 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932336    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932339    6116 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932341    6116 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932346    6116 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932350    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932354    6116 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932358    6116 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932361    6116 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:35:14.946542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932365    6116 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:35:14.947152 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.932410    6116 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:35:14.947152 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932656    6116 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:35:14.947152 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932667    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:35:14.947152 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932672    6116 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:35:14.947152 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932676    6116 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:35:14.947152 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932680    6116 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:35:14.947152 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932685    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:35:14.947152 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932689    6116 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:35:14.947152 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932694    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:35:14.947152 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932701    6116 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:35:14.947152 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932707    6116 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:35:14.947152 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932712    6116 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:35:14.947152 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932715    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:35:14.947152 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932720    6116 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:35:14.947152 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932724    6116 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932728    6116 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932732    6116 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932737    6116 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932741    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932746    6116 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932750    6116 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932754    6116 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932759    6116 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932763    6116 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932767    6116 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932772    6116 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932777    6116 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932781    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932785    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932789    6116 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932793    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932798    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932802    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932806    6116 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:35:14.947608 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932810    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932814    6116 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932818    6116 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932822    6116 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932826    6116 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932830    6116 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932836    6116 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932841    6116 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932845    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932849    6116 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932853    6116 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932857    6116 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932861    6116 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932866    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932869    6116 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932873    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932877    6116 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932882    6116 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932886    6116 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:35:14.948185 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932890    6116 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932894    6116 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932899    6116 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932903    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932907    6116 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932912    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932915    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932920    6116 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932925    6116 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932928    6116 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932933    6116 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932937    6116 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932942    6116 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932946    6116 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932950    6116 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932955    6116 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932958    6116 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932963    6116 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932967    6116 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:35:14.948805 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932971    6116 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932975    6116 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932979    6116 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932983    6116 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932987    6116 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932991    6116 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932995    6116 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.932999    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933003    6116 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933007    6116 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933011    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933015    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933019    6116 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933023    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933027    6116 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933031    6116 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933037    6116 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933042    6116 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933046    6116 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933050    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:35:14.949327 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933054    6116 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:35:14.949902 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933060    6116 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:35:14.949902 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933065    6116 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:35:14.949902 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933069    6116 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:35:14.949902 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933073    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:35:14.949902 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933078    6116 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:35:14.949902 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933082    6116 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:35:14.949902 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933085    6116 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:35:14.949902 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933090    6116 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:35:14.949902 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933093    6116 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:35:14.949902 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933097    6116 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:35:14.949902 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933101    6116 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:35:14.949902 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933105    6116 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:35:14.949902 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933109    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:35:14.949902 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933113    6116 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:35:14.949902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.933159    6116 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 01:35:14.949902 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933213    6116 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933220    6116 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933224    6116 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933229    6116 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933233    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933238    6116 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933242    6116 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933247    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933251    6116 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933255    6116 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933259    6116 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933264    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933269    6116 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933273    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933278    6116 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933283    6116 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933291    6116 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933297    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933306    6116 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 01:35:14.950391 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933310    6116 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933313    6116 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933316    6116 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933318    6116 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933321    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933323    6116 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933325    6116 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933328    6116 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933330    6116 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933333    6116 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933335    6116 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933338    6116 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933341    6116 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933343    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933346    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933348    6116 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933351    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933354    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933357    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933359    6116 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 01:35:14.951059 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933361    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933363    6116 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933366    6116 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933369    6116 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933371    6116 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933373    6116 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933377    6116 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933381    6116 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933384    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933387    6116 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933389    6116 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933394    6116 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933396    6116 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933400    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933402    6116 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933406    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933411    6116 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933415    6116 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933419    6116 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 01:35:14.951694 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933423    6116 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933426    6116 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933429    6116 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933431    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933434    6116 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933436    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933439    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933441    6116 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933468    6116 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933473    6116 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933477    6116 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933483    6116 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933488    6116 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933492    6116 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933495    6116 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933498    6116 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933500    6116 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933503    6116 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933505    6116 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 01:35:14.952303 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933511    6116 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933515    6116 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933521    6116 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933526    6116 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933530    6116 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933536    6116 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933540    6116 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933545    6116 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933548    6116 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933551    6116 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933553    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933556    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933560    6116 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933564    6116 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933568    6116 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933572    6116 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933576    6116 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933581    6116 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933585    6116 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933590    6116 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 01:35:14.953062 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933594    6116 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 01:35:14.953762 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933599    6116 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 01:35:14.953762 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933603    6116 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 01:35:14.953762 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933608    6116 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 01:35:14.953762 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933611    6116 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 01:35:14.953762 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933616    6116 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 01:35:14.953762 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933620    6116 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 01:35:14.953762 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933624    6116 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 01:35:14.953762 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:14.933628    6116 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 01:35:14.953762 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.933721    6116 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 01:35:14.953762 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.935199    6116 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 04 01:35:14.953762 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.935271    6116 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 04 01:35:14.953762 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.935549    6116 server.go:1014] "Starting client certificate rotation"
Nov 04 01:35:14.953762 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.935670    6116 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 01:35:14.953762 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.935730    6116 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 01:29:02 +0000 UTC" deadline="2028-08-04 10:21:29.233073666 +0000 UTC"
Nov 04 01:35:14.953762 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.935768    6116 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15344h46m14.297307697s"
Nov 04 01:35:14.953762 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.936268    6116 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:35:14.954261 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.937948    6116 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 01:35:14.954261 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.939402    6116 log.go:25] "Validated CRI v1 runtime API"
Nov 04 01:35:14.954261 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.941995    6116 log.go:25] "Validated CRI v1 image API"
Nov 04 01:35:14.954261 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.942933    6116 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 01:35:14.954261 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.946254    6116 fs.go:135] Filesystem UUIDs: map[308bd465-3a5b-422e-b29a-7f0c430bde57:/dev/nvme0n1p3 666f8308-26f6-4156-8c6b-aab85c4569c0:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Nov 04 01:35:14.954499 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.946284    6116 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1ff7d6b1c20e0f6b1137ac080e30bd0af5e1da81a648bf34794afb2c8ec46bc1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1ff7d6b1c20e0f6b1137ac080e30bd0af5e1da81a648bf34794afb2c8ec46bc1/userdata/shm major:0 minor:206 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2a3a6f723c4544bbd5e53e1828f49093eda93bb90a92e7449075831c3392917d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2a3a6f723c4544bbd5e53e1828f49093eda93bb90a92e7449075831c3392917d/userdata/shm major:0 minor:65 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2f3c96e1085f141b75bcef65fb3d5f5f9605cfd40e3718e1989ec632be023f3e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2f3c96e1085f141b75bcef65fb3d5f5f9605cfd40e3718e1989ec632be023f3e/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/355a1c2d04ae568f6664e694146974b8ab351c306ea8bb7e39207174d85a98bf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/355a1c2d04ae568f6664e694146974b8ab351c306ea8bb7e39207174d85a98bf/userdata/shm major:0 minor:354 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/35ddf6e87039025375b3fb74fa96171ce2e40a076548a59380f15444aef36a7f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/35ddf6e87039025375b3fb74fa96171ce2e40a076548a59380f15444aef36a7f/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/37d0f2edd1924684ff91503d18feac64b6b4ff2bc8599be69f80de3c41c9fccc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/37d0f2edd1924684ff91503d18feac64b6b4ff2bc8599be69f80de3c41c9fccc/userdata/shm major:0 minor:329 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3f0fb6f9a3020f5743953f382e19d597cb410dba4214a2216fc9faf51dfab262/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3f0fb6f9a3020f5743953f382e19d597cb410dba4214a2216fc9faf51dfab262/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/40959d56f16d23772a65b94dc695188164936b718ccfb8aaca0441818f2fb587/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/40959d56f16d23772a65b94dc695188164936b718ccfb8aaca0441818f2fb587/userdata/shm major:0 minor:271 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/46bc5e27140f7feda0532059508a8f407c36ea862d2005918b83dd88f2b0c7f6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/46bc5e27140f7feda0532059508a8f407c36ea862d2005918b83dd88f2b0c7f6/userdata/shm major:0 minor:317 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/52aacf3160fabbfbae094dfd665f1b3e17861bd327f8330ff8be7902ee822456/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/52aacf3160fabbfbae094dfd665f1b3e17861bd327f8330ff8be7902ee822456/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5471b6fa46f5222a9d35b493eaece92ee03c1f1d0b93043685e526dfe5784179/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5471b6fa46f5222a9d35b493eaece92ee03c1f1d0b93043685e526dfe5784179/userdata/shm major:0 minor:63 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6f74ad59227901ed5d82dbb766f8ab4258717c3beb210c4cfa68d3b575b27ec3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6f74ad59227901ed5d82dbb766f8ab4258717c3beb210c4cfa68d3b575b27ec3/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/74b1b2138f9f936a36d0e5a89ca49ec63b1354e0f13379fdb892bdc26b16b2c5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/74b1b2138f9f936a36d0e5a89ca49ec63b1354e0f13379fdb892bdc26b16b2c5/userdata/shm major:0 minor:62 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7e89eb172b9b14a9b75ad94924e7d50441b3927e6200f523840e9345b97d205e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7e89eb172b9b14a9b75ad94924e7d50441b3927e6200f523840e9345b97d205e/userdata/shm major:0 minor:276 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/84fb8fd6e9424f9f7ce12f856c19373e6bee5a388fedaa3eabd1636fa96c6389/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/84fb8fd6e9424f9f7ce12f856c19373e6bee5a388fedaa3eabd1636fa96c6389/userdata/shm major:0 minor:64 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/86205162f356bdf692e2483138eb9e186d5d9f994d51544b807266900e53264f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/86205162f356bdf692e2483138eb9e186d5d9f994d51544b807266900e53264f/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8a44dae6456170696ad7634257802f637ee6dbc19006508e859beaca7e039bb9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8a44dae6456170696ad7634257802f637ee6dbc19006508e859beaca7e039bb9/userdata/shm major:0 minor:334 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8c41e311573c0941f5600843942d61c63ceeece2bbd20c365768575c3b2430f8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8c41e311573c0941f5600843942d61c63ceeece2bbd20c365768575c3b2430f8/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b4fb007454368a64771fd1e72d20bcd0cb130ed0877b8d84d9f479d4f4f3b0d8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b4fb007454368a64771fd1e72d20bcd0cb130ed0877b8d84d9f479d4f4f3b0d8/userdata/shm major:0 minor:331 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c9933ed453da9a8ee31ec474c134862f5d132629cfd5b1f20aca266bcd74b75a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c9933ed453da9a8ee31ec474c134862f5d132629cfd5b1f20aca266bcd74b75a/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d96830bd110f9ffe63a8f5c97af594eba61bab50ce0382a4e784b76c72dcc8bd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d96830bd110f9ffe63a8f5c97af594eba61bab50ce0382a4e784b76c72dcc8bd/userdata/shm major:0 minor:310 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e7daeb00d50a5744358d82d076f8b8225059b60a1f030e4c13e7672a968948fa/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e7daeb00d50a5744358d82d076f8b8225059b60a1f030e4c13e7672a968948fa/userdata/shm major:0 minor:204 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/010e4cd1-587e-43a7-9f58-93ca75a074df/volumes/kubernetes.io~projected/kube-api-access-4nrq5:{mountpoint:/var/lib/kubelet/pods/010e4cd1-587e-43a7-9f58-93ca75a074df/volumes/kubernetes.io~projected/kube-api-access-4nrq5 major:0 minor:316 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/010e4cd1-587e-43a7-9f58-93ca75a074df/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/010e4cd1-587e-43a7-9f58-93ca75a074df/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:315 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/010e4cd1-587e-43a7-9f58-93ca75a074df/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/010e4cd1-587e-43a7-9f58-93ca75a074df/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:314 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc/volumes/kubernetes.io~projected/kube-api-access-28b8j:{mountpoint:/var/lib/kubelet/pods/0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc/volumes/kubernetes.io~projected/kube-api-access-28b8j major:0 minor:309 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:305 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:323 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/192e2fec-fefb-4bc5-9516-ed7464963a10/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/192e2fec-fefb-4bc5-9516-ed7464963a10/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/192e2fec-fefb-4bc5-9516-ed7464963a10/volumes/kubernetes.io~projected/kube-api-access-gxtbz:{mountpoint:/var/lib/kubelet/pods/192e2fec-fefb-4bc5-9516-ed7464963a10/volumes/kubernetes.io~projected/kube-api-access-gxtbz major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/192e2fec-fefb-4bc5-9516-ed7464963a10/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/192e2fec-fefb-4bc5-9516-ed7464963a10/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1e5c731f-7d75-42d6-8ebd-ee742632fa3d/volumes/kubernetes.io~projected/kube-api-access-5fjld:{mountpoint:/var/lib/kubelet/pods/1e5c731f-7d75-42d6-8ebd-ee742632fa3d/volumes/kubernetes.io~projected/kube-api-access-5fjld major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1e5c731f-7d75-42d6-8ebd-ee742632fa3d/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/1e5c731f-7d75-42d6-8ebd-ee742632fa3d/volumes/kubernetes.io~secret/cert major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/20296e8e-e25f-40e1-9b2b-4971b7e69141/volumes/kubernetes.io~projected/kube-api-access-58s5t:{mountpoint:/var/lib/kubelet/pods/20296e8e-e25f-40e1-9b2b-4971b7e69141/volumes/kubernetes.io~projected/kube-api-access-58s5t major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/37d34756-13c3-4993-a3d1-07f68eada6b0/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/37d34756-13c3-4993-a3d1-07f68eada6b0/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/37d34756-13c3-4993-a3d1-07f68eada6b0/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/37d34756-13c3-4993-a3d1-07f68eada6b0/volumes/kubernetes.io~empty-dir/tmp major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/37d34756-13c3-4993-a3d1-07f68eada6b0/volumes/kubernetes.io~projected/kube-api-access-xvlmf:{mountpoint:/var/lib/kubelet/pods/37d34756-13c3-4993-a3d1-07f68eada6b0/volumes/kubernetes.io~projected/kube-api-access-xvlmf major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f/volumes/kubernetes.io~projected/kube-api-access-4fqmn:{mountpoint:/var/lib/kubelet/pods/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f/volumes/kubernetes.io~projected/kube-api-access-4fqmn major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f/volumes/kubernetes.io~secret/metrics-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/469986a0-2b14-47a4-8f4d-3a1456447cdb/volumes/kubernetes.io~projected/kube-api-access-4nrqq:{mountpoint:/var/lib/kubelet/pods/469986a0-2b14-47a4-8f4d-3a1456447cdb/volumes/kubernetes.io~projected/kube-api-access-4nrqq major:0 minor:270 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/469986a0-2b14-47a4-8f4d-3a1456447cdb/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/469986a0-2b14-47a4-8f4d-3a1456447cdb/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:269 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/469986a0-2b14-47a4-8f4d-3a1456447cdb/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/469986a0-2b14-47a4-8f4d-3a1456447cdb/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:265 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/480e250d-44a3-4d5c-987d-22737107544e/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/480e250d-44a3-4d5c-987d-22737107544e/volumes/kubernetes.io~secret/agent-certs major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f/volumes/kubernetes.io~projected/kube-api-access-lrqc6:{mountpoint:/var/lib/kubelet/pods/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f/volumes/kubernetes.io~projected/kube-api-access-lrqc6 major:0 minor:275 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:274 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:273 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52cfdb11-898a-4673-97c5-ecddc1385a03/volumes/kubernetes.io~projected/kube-api-access-m6fjs:{mountpoint:/var/lib/kubelet/pods/52cfdb11-898a-4673-97c5-ecddc1385a03/volumes/kubernetes.io~projected/kube-api-access-m6fjs major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/67ed6df2-450d-4705-8a11-f98d5b507dc4/volumes/kubernetes.io~projected/kube-api-access-gxtjk:{mountpoint:/var/lib/kubelet/pods/67ed6df2-450d-4705-8a11-f98d5b507dc4/volumes/kubernetes.io~projected/kube-api-access-gxtjk major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7acf7f6e-efb8-498e-8066-a60fbedecd49/volumes/kubernetes.io~projected/kube-api-access-4vz9t:{mountpoint:/var/lib/kubelet/pods/7acf7f6e-efb8-498e-8066-a60fbedecd49/volumes/kubernetes.io~projected/kube-api-access-4vz9t major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a8f004fb-8c9c-4efb-b60d-358e32ba5120/volumes/kubernetes.io~projected/kube-api-access-7647t:{mountpoint:/var/lib/kubelet/pods/a8f004fb-8c9c-4efb-b60d-358e32ba5120/volumes/kubernetes.io~projected/kube-api-access-7647t major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aa398d49-e1ee-41df-8785-b765e1f35b18/volumes/kubernetes.io~projected/kube-api-access-lzftg:{mountpoint:/var/lib/kubelet/pods/aa398d49-e1ee-41df-8785-b765e1f35b18/volumes/kubernetes.io~projected/kube-api-access-lzftg major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ca08235e-943c-4d8d-939d-4ceaedd9cf92/volumes/kubernetes.io~projected/kube-api-access-7c8rj:{mountpoint:/var/lib/kubelet/pods/ca08235e-943c-4d8d-939d-4ceaedd9cf92/volumes/kubernetes.io~projected/kube-api-access-7c8rj major:0 minor:333 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce/volumes/kubernetes.io~projected/kube-api-access-lls22:{mountpoint:/var/lib/kubelet/pods/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce/volumes/kubernetes.io~projected/kube-api-access-lls22 major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:348 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~empty-dir/config-out major:0 minor:347 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~projected/kube-api-access-v72sb:{mountpoint:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~projected/kube-api-access-v72sb major:0 minor:353 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~projected/tls-assets major:0 minor:346 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/config major:0 minor:342 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:352 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:344 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:351 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:350 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:343 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:345 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:349 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/web-config major:0 minor:348 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e62bee01-e717-4484-bb3d-ff4af522a355/volumes/kubernetes.io~projected/kube-api-access-4gmb7:{mountpoint:/var/lib/kubelet/pods/e62bee01-e717-4484-bb3d-ff4af522a355/volumes/kubernetes.io~projected/kube-api-access-4gmb7 major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e62bee01-e717-4484-bb3d-ff4af522a355/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/e62bee01-e717-4484-bb3d-ff4af522a355/volumes/kubernetes.io~secret/metrics-certs major:0 minor:328 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb/volumes/kubernetes.io~projected/kube-api-access-dq7r6:{mountpoint:/var/lib/kubelet/pods/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb/volumes/kubernetes.io~projected/kube-api-access-dq7r6 major:0 minor:241 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:240 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/dfd3aa3216ba8310590c0e4209a967f04cae581610fd02aae2df855d6dabb17f/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/e2ac3c4bb32d3c34e5d89aa4686d572f29bcda802b7d1abba5973ca76f9bf3c6/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/c576d493b4c77f8a09d0f010026d35b824f25b59127a740f5f80791e53cb7c56/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-108:{mountpoint:/var/lib/containers/storage/overlay/c0dd97dbe22714a8ebd9904e170e62afbfb9b6ab09e6030c11b51c258c6d032d/merged major:0 minor:108 fsType:overlay blockSize:0} overlay_0-111:{mountpoint:/var/lib/containers/storage/overlay/fca98645cd49aad99604ed6d555c06495d5219acd4bc1cde18a8d97c07b2603b/merged major:0 minor:111 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/3f1ff3c11461cdb053bd90a014d63a3224e7a30a9592189875dafe93ed0f8797/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/037f5e583f64f929325d84568089c52213fa70111238774d387950022e676044/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/ddda8d91888a99a7f626cbc00d906463f8b1dee172a352fc32358fee5bfb5ef8/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/8c3485c1430bc772b27d52576dbfd12db87356a35dbeaf160ee0cdbbb9961962/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/ba399d62acbaec1d3d6a50678d2317b89066073b9a8a3627c70e02491fdc145e/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/10bf57af0867a6130c3c62f21f5f6a64aa6b32d2785e5dd5b2dbd3c5b7c38f46/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/830a9a38bcc4a5453ec1bfb9831b8099301e75ae8ddd10c9c8d73a5dfe197ed8/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/b3ff5c89e14ef9974d6ab3984cf9ffb21ade84362af179e6b55aad5285550c25/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/b9d60398fb5b2e469258a723e9087f8202a15c9ea22e176363138a8a091894df/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/3ab55c5b7165c1034408a2ab9c37dfa4de35109a109a039ec6f51472962d0ee0/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/21a4ba8cb85676fb0b715290ddfc750ed4824983106c3f3d6e89a23924139ced/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/b4de4b4917388a09a87520955afe5ea059a5fc757bb9014c8a8d0d1903396b59/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/fdf2fe31e5a97395d55922c57497ffcf6cffebff297dfd4e46167ed8dd692158/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/95c6d0aa568a5d22bed23d913775c8a9f50761a076f08cfac2fedaa4236288fc/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/eca5db73c72d65e4c4b19884e94eff65443ecf80760f34064d7a0307c499471d/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/0cfcc1e81e3c5201218cfc82e4bfc7e29dca58cc77a6d28b024ad3c82de5210b/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/f758e2a7edb4fbad6b3859fe2e669bc1ddc30c4bc374a63ec2f724b8a7476e91/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/95b76d0cb48df73dc7ef8f6c408c861379a1db996253f6a05296d9639edb1b1d/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/095eb4a5f21ee12fc4285813d2ccfa813766e4cc3f97f6d708d8156471e834eb/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/7de7c48cb20cc653ff72678153aceaad705ce172bf869b0de9b210ebc88f6546/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/c9989df887cad64a9e23c72da1c82885afcb9fabf4d35ffa6d9fa5445f35a83f/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/025e42b2ca5a940ccbc7d8085bfcddf59cb7340cc9450a2e0a2fca42f0837b5b/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/979e4cf97cab369719ac84506023985912f63eb01560b95a306db6adcd7774d3/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-244:{mountpoint:/var/lib/containers/storage/overlay/254b1038eb535ac9691a392bb0705d9277a259d2567c1c6a414a22acdca61775/merged major:0 minor:244 fsType:overlay blockSize:0} overlay_0-246:{mountpoint:/var/lib/containers/storage/overlay/c4b5853dc17ccdd0e8cef5574457d95f13ee2bf8629c73d155e4ea4de6c5afde/merged major:0 minor:246 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/ab556d9d4281c33befb2d4adf5bae52eb4cd4cabc0156da02a96fe9ba2b0b739/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-257:{mountpoint:/var/lib/containers/storage/overlay/ebfabc49bf291f690de8c777dbedc6be1cec714d3c8548524812c25b52944052/merged major:0 minor:257 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/373f96f2b90a6f2215db3c6236e1f8181950abe38b500374b35b4b9a2c335f49/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-278:{mountpoint:/var/lib/containers/storage/overlay/32633f244edfcfeb18ca10c2e1a1f44d9cb95b449d3c60f8177a02512f14e979/merged major:0 minor:278 fsType:overlay blockSize:0} overlay_0-280:{mountpoint:/var/lib/containers/storage/overlay/f66a6b495904005932afa2f87cbb60e8b27159e78673d73e8f2f83052d99a885/merged major:0 minor:280 fsType:overlay blockSize:0} overlay_0-282:{mountpoint:/var/lib/containers/storage/overlay/720c8dcfc2809a824d0c6385bda2889811955ccd764ccd428077361dc2821483/merged major:0 minor:282 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/0a1fef0b879da4b2343a67e7fdcd2514040e99250e9051068b73f835dbb264b0/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-290:{mountpoint:/var/lib/containers/storage/overlay/1b240795f1cd48c9ca97c812996b3d5a8b88b46fc2dac2a3869e5023e680b9d8/merged major:0 minor:290 fsType:overlay blockSize:0} overlay_0-295:{mountpoint:/var/lib/containers/storage/overlay/359ca5ba5ecd06f3c22fab8d3b0015c53a563005e9077d3fc847f514974df8a2/merged major:0 minor:295 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/28a909f7bece15f5e562c067b15f20d0cdbfce46178362474bd64211b5b84961/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/e47fd32a3b53e667308bb8e4050cf4ebf18e660e3d04edb2f5b4ac91f0a9ef24/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-319:{mountpoint:/var/lib/containers/storage/overlay/870076d8b9f22c4ecfe3495f5ecb245e3d3fcc3f7379beb30ae190651397cfb1/merged major:0 minor:319 fsType:overlay blockSize:0} overlay_0-321:{mountpoint:/var/lib/containers/storage/overlay/cfa17ab2c945e495b682d2814018dd8d35bccfc55665b2b838a630ddd7773989/merged major:0 minor:321 fsType:overlay blockSize:0} overlay_0-336:{mountpoint:/var/lib/containers/storage/overlay/d94604911d512c02d5ea3a69b1efb84d9b228851c87d1dcf1d0ae7063ef476ec/merged major:0 minor:336 fsType:overlay blockSize:0} overlay_0-338:{mountpoint:/var/lib/containers/storage/overlay/40472d9a0d92ff7085615e884e85ff49f7e67a832634d11c3c658ad03adc3c99/merged major:0 minor:338 fsType:overlay blockSize:0} overlay_0-340:{mountpoint:/var/lib/containers/storage/overlay/fd34ef922c92a52ad2f3cd45235ec0af4d398ed793839659f056d2d7c7e30be8/merged major:0 minor:340 fsType:overlay blockSize:0} overlay_0-356:{mountpoint:/var/lib/containers/storage/overlay/167d7ca1d204c8294607fa0af2c2374628f68177c0ac636a9a721ab89e183ace/merged major:0 minor:356 fsType:overlay blockSize:0} overlay_0-358:{mountpoint:/var/lib/containers/storage/overlay/54929587bbc720a75db1bd4017680186be6e768510272b553fb2ae643c193029/merged major:0 minor:358 fsType:overlay blockSize:0} overlay_0-363:{mountpoint:/var/lib/containers/storage/overlay/2ff8e54b5755448246a5cbab5eb100b8ca8ada0b5aca17547ae448c8b4cd6708/merged major:0 minor:363 fsType:overlay blockSize:0} overlay_0-366:{mountpoint:/var/lib/containers/storage/overlay/c0db2152ece126bf1da0e1d4c61498a8f8a82f7785b8097f41fb9316468f53fe/merged major:0 minor:366 fsType:overlay blockSize:0} overlay_0-371:{mountpoint:/var/lib/containers/storage/overlay/5b65e063644d7377b3a04ca89cc8691ec5658715743a9df6c896d2f2ef98cf18/merged major:0 minor:371 fsType:overlay blockSize:0} overlay_0-377:{mountpoint:/var/lib/containers/storage/overlay/478f6ff50a91513351e5edf0af0fab3822ab6ac331d9f0a1ca4dd03a77dc4576/merged major:0 minor:377 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/2a76ee8966b4307b73358c0a98d73fa683159f3c467d61d5d625c8ca865eacd2/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/a12ec536a5ccd65956a42c1436fe62f008135cd7fef51b1e4a5620aff1a4df7a/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/0591737aa5d0ccd7fa66af371bbc7d2eec1875701d96a3152b6a2ffd068eaf94/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/56471ef204034ce2b1bd6bb3ee3d2aba4e25df4326b873bfb1e94c6bd596f700/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/b331ab0fa98dd96f79ceade83ee2dad9a762ae14d951c59404f93d00078e0f9a/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/9f7fb2573e4e561032328cb225528f30ddd82fa79faf80cb83a94778af4164f6/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/b692adeb9236338d0e54e1a3f015401652a7bc173cb1f856dc1771dcb54fca0a/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/bb6147a179deb92a2fe3345345cfcd180682416cebad1a90ea203fa265dc1f41/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/f6391ff9a7244e8ec007b8a6a275d17c74c0b9b4e9b7a3ac8a6b3cc2121a8433/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/9db7317b9ad5987b80f38981cc3d81b220fe950f37b236e598a9df32fe991f8b/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 04 01:35:14.954962 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.954493    6116 manager.go:217] Machine: {Timestamp:2026-11-04 01:35:14.953893013 +0000 UTC m=+0.075170530 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200379 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2463b48f9f5bacb0d3436e9d663aae SystemUUID:ec2463b4-8f9f-5bac-b0d3-436e9d663aae BootID:ceb81161-b925-4d65-b9e8-bc933ac39cd8 Filesystems:[{Device:overlay_0-336 DeviceMajor:0 DeviceMinor:336 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-371 DeviceMajor:0 DeviceMinor:371 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/86205162f356bdf692e2483138eb9e186d5d9f994d51544b807266900e53264f/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/010e4cd1-587e-43a7-9f58-93ca75a074df/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:315 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/52cfdb11-898a-4673-97c5-ecddc1385a03/volumes/kubernetes.io~projected/kube-api-access-m6fjs DeviceMajor:0 DeviceMinor:55 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:351 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8c41e311573c0941f5600843942d61c63ceeece2bbd20c365768575c3b2430f8/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/355a1c2d04ae568f6664e694146974b8ab351c306ea8bb7e39207174d85a98bf/userdata/shm DeviceMajor:0 DeviceMinor:354 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-356 DeviceMajor:0 DeviceMinor:356 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc/volumes/kubernetes.io~projected/kube-api-access-28b8j DeviceMajor:0 DeviceMinor:309 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-319 DeviceMajor:0 DeviceMinor:319 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/20296e8e-e25f-40e1-9b2b-4971b7e69141/volumes/kubernetes.io~projected/kube-api-access-58s5t DeviceMajor:0 DeviceMinor:56 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/84fb8fd6e9424f9f7ce12f856c19373e6bee5a388fedaa3eabd1636fa96c6389/userdata/shm DeviceMajor:0 DeviceMinor:64 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/010e4cd1-587e-43a7-9f58-93ca75a074df/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:314 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/480e250d-44a3-4d5c-987d-22737107544e/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:50 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/35ddf6e87039025375b3fb74fa96171ce2e40a076548a59380f15444aef36a7f/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:273 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-282 DeviceMajor:0 DeviceMinor:282 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/46bc5e27140f7feda0532059508a8f407c36ea862d2005918b83dd88f2b0c7f6/userdata/shm DeviceMajor:0 DeviceMinor:317 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/74b1b2138f9f936a36d0e5a89ca49ec63b1354e0f13379fdb892bdc26b16b2c5/userdata/shm DeviceMajor:0 DeviceMinor:62 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c9933ed453da9a8ee31ec474c134862f5d132629cfd5b1f20aca266bcd74b75a/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:45 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1e5c731f-7d75-42d6-8ebd-ee742632fa3d/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:197 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/469986a0-2b14-47a4-8f4d-3a1456447cdb/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:265 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ca08235e-943c-4d8d-939d-4ceaedd9cf92/volumes/kubernetes.io~projected/kube-api-access-7c8rj DeviceMajor:0 DeviceMinor:333 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-366 DeviceMajor:0 DeviceMinor:366 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-290 DeviceMajor:0 DeviceMinor:290 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:305 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-340 DeviceMajor:0 DeviceMinor:340 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:342 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:350 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:348 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/192e2fec-fefb-4bc5-9516-ed7464963a10/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:48 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1e5c731f-7d75-42d6-8ebd-ee742632fa3d/volumes/kubernetes.io~projected/kube-api-access-5fjld DeviceMajor:0 DeviceMinor:202 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/469986a0-2b14-47a4-8f4d-3a1456447cdb/volumes/kubernetes.io~projected/kube-api-access-4nrqq DeviceMajor:0 DeviceMinor:270 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:343 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:352 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/37d34756-13c3-4993-a3d1-07f68eada6b0/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:49 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7acf7f6e-efb8-498e-8066-a60fbedecd49/volumes/kubernetes.io~projected/kube-api-access-4vz9t DeviceMajor:0 DeviceMinor:58 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2f3c96e1085f141b75bcef65fb3d5f5f9605cfd40e3718e1989ec632be023f3e/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3f0fb6f9a3020f5743953f382e19d597cb410dba4214a2216fc9faf51dfab262/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:344 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:348 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1ff7d6b1c20e0f6b1137ac080e30bd0af5e1da81a648bf34794afb2c8ec46bc1/userdata/shm DeviceMajor:0 DeviceMinor:206 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-246 DeviceMajor:0 DeviceMinor:246 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8a44dae6456170696ad7634257802f637ee6dbc19006508e859beaca7e039bb9/userdata/shm DeviceMajor:0 DeviceMinor:334 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:349 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5471b6fa46f5222a9d35b493eaece92ee03c1f1d0b93043685e526dfe5784179/userdata/shm DeviceMajor:0 DeviceMinor:63 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:240 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/52aacf3160fabbfbae094dfd665f1b3e17861bd327f8330ff8be7902ee822456/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a8f004fb-8c9c-4efb-b60d-358e32ba5120/volumes/kubernetes.io~projected/kube-api-access-7647t DeviceMajor:0 DeviceMinor:52 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-280 DeviceMajor:0 DeviceMinor:280 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:274 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-295 DeviceMajor:0 DeviceMinor:295 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/37d34756-13c3-4993-a3d1-07f68eada6b0/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:47 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b4fb007454368a64771fd1e72d20bcd0cb130ed0877b8d84d9f479d4f4f3b0d8/userdata/shm DeviceMajor:0 DeviceMinor:331 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-108 DeviceMajor:0 DeviceMinor:108 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f/volumes/kubernetes.io~projected/kube-api-access-lrqc6 DeviceMajor:0 DeviceMinor:275 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:346 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-358 DeviceMajor:0 DeviceMinor:358 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6f74ad59227901ed5d82dbb766f8ab4258717c3beb210c4cfa68d3b575b27ec3/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d96830bd110f9ffe63a8f5c97af594eba61bab50ce0382a4e784b76c72dcc8bd/userdata/shm DeviceMajor:0 DeviceMinor:310 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e7daeb00d50a5744358d82d076f8b8225059b60a1f030e4c13e7672a968948fa/userdata/shm DeviceMajor:0 DeviceMinor:204 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb/volumes/kubernetes.io~projected/kube-api-access-dq7r6 DeviceMajor:0 DeviceMinor:241 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-257 DeviceMajor:0 DeviceMinor:257 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7e89eb172b9b14a9b75ad94924e7d50441b3927e6200f523840e9345b97d205e/userdata/shm DeviceMajor:0 DeviceMinor:276 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-377 DeviceMajor:0 DeviceMinor:377 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e62bee01-e717-4484-bb3d-ff4af522a355/volumes/kubernetes.io~projected/kube-api-access-4gmb7 DeviceMajor:0 DeviceMinor:51 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/37d34756-13c3-4993-a3d1-07f68eada6b0/volumes/kubernetes.io~projected/kube-api-access-xvlmf DeviceMajor:0 DeviceMinor:53 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:345 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2a3a6f723c4544bbd5e53e1828f49093eda93bb90a92e7449075831c3392917d/userdata/shm DeviceMajor:0 DeviceMinor:65 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-321 DeviceMajor:0 DeviceMinor:321 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/37d0f2edd1924684ff91503d18feac64b6b4ff2bc8599be69f80de3c41c9fccc/userdata/shm DeviceMajor:0 DeviceMinor:329 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/192e2fec-fefb-4bc5-9516-ed7464963a10/volumes/kubernetes.io~projected/kube-api-access-gxtbz DeviceMajor:0 DeviceMinor:59 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/192e2fec-fefb-4bc5-9516-ed7464963a10/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/aa398d49-e1ee-41df-8785-b765e1f35b18/volumes/kubernetes.io~projected/kube-api-access-lzftg DeviceMajor:0 DeviceMinor:54 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-111 DeviceMajor:0 DeviceMinor:111 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce/volumes/kubernetes.io~projected/kube-api-access-lls22 DeviceMajor:0 DeviceMinor:205 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/40959d56f16d23772a65b94dc695188164936b718ccfb8aaca0441818f2fb587/userdata/shm DeviceMajor:0 DeviceMinor:271 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/010e4cd1-587e-43a7-9f58-93ca75a074df/volumes/kubernetes.io~projected/kube-api-access-4nrq5 DeviceMajor:0 DeviceMinor:316 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:323 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-338 DeviceMajor:0 DeviceMinor:338 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f/volumes/kubernetes.io~projected/kube-api-access-4fqmn DeviceMajor:0 DeviceMinor:201 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:236 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:347 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes/kubernetes.io~projected/kube-api-access-v72sb DeviceMajor:0 DeviceMinor:353 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-363 DeviceMajor:0 DeviceMinor:363 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-244 DeviceMajor:0 DeviceMinor:244 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-278 DeviceMajor:0 DeviceMinor:278 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e62bee01-e717-4484-bb3d-ff4af522a355/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:328 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/67ed6df2-450d-4705-8a11-f98d5b507dc4/volumes/kubernetes.io~projected/kube-api-access-gxtjk DeviceMajor:0 DeviceMinor:57 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:203 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/469986a0-2b14-47a4-8f4d-3a1456447cdb/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:269 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:1ff7d6b1c20e0f6 MacAddress:46:6b:5e:65:6d:33 Speed:10000 Mtu:8901} {Name:355a1c2d04ae568 MacAddress:b6:da:36:4a:04:c0 Speed:10000 Mtu:8901} {Name:35ddf6e87039025 MacAddress:0e:d7:e0:b5:ae:14 Speed:10000 Mtu:8901} {Name:37d0f2edd192468 MacAddress:06:87:13:de:15:75 Speed:10000 Mtu:8901} {Name:40959d56f16d237 MacAddress:fe:4a:a0:6c:54:0a Speed:10000 Mtu:8901} {Name:46bc5e27140f7fe MacAddress:82:71:c0:c2:e0:3f Speed:10000 Mtu:8901} {Name:86205162f356bdf MacAddress:5a:a1:33:88:4d:fb Speed:10000 Mtu:8901} {Name:8a44dae64561706 MacAddress:5a:1f:f8:26:c8:0d Speed:10000 Mtu:8901} {Name:b4fb007454368a6 MacAddress:9e:ac:61:70:94:da Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:72:78:8a:cd:b3 Speed:0 Mtu:9001} {Name:br-int MacAddress:ea:ce:14:8b:4e:76 Speed:0 Mtu:8901} {Name:d96830bd110f9ff MacAddress:be:e2:a0:0b:d8:22 Speed:10000 Mtu:8901} {Name:e7daeb00d50a574 MacAddress:9a:9b:15:83:f0:8c Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:72:78:8a:cd:b3 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:8e:d7:9c:eb:d8:5a Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:56:b2:6c:cd:bf:d4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 01:35:14.955188 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.954973    6116 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 01:35:14.955188 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.955046    6116 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 01:35:14.955425 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.955396    6116 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 01:35:14.955648 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.955428    6116 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-138.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 01:35:14.955648 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.955626    6116 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 01:35:14.955648 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.955638    6116 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 01:35:14.955803 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.955650    6116 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:35:14.955803 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.955676    6116 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 01:35:14.955856 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.955802    6116 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 01:35:14.956027 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.956006    6116 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 01:35:14.956179 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.956156    6116 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 01:35:14.956222 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.956193    6116 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 01:35:14.956222 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.956208    6116 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 01:35:14.956222 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.956218    6116 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 01:35:14.956344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.956229    6116 kubelet.go:405] "Adding apiserver pod source"
Nov 04 01:35:14.956344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.956241    6116 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 01:35:14.957022 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.957004    6116 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 01:35:14.957022 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.957020    6116 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:35:14.957825 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.957804    6116 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 01:35:14.970283 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.970255    6116 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 01:35:14.970283 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.970280    6116 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 01:35:14.970522 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.970422    6116 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 04 01:35:14.970696 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.970681    6116 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 01:35:14.970775 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.970729    6116 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 01:35:14.970885 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.970872    6116 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 01:35:14.970965 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.970893    6116 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 01:35:14.970965 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.970902    6116 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 01:35:14.970965 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.970911    6116 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 01:35:14.970965 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.970919    6116 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 01:35:14.970965 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.970930    6116 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 01:35:14.970965 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.970938    6116 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 01:35:14.970965 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.970945    6116 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 01:35:14.970965 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.970957    6116 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 01:35:14.970965 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.970966    6116 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 01:35:14.971305 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.970988    6116 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 01:35:14.971305 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.971004    6116 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 01:35:14.971305 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.971083    6116 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 01:35:14.971305 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.971094    6116 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 01:35:14.975172 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.975150    6116 server.go:1263] "Started kubelet"
Nov 04 01:35:14.975344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.975310    6116 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 01:35:14.975628 ip-10-0-129-138 systemd[1]: Started Kubernetes Kubelet.
Nov 04 01:35:14.980118 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.977885    6116 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 01:35:14.980118 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.978012    6116 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 01:35:14.980118 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.978852    6116 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 01:35:14.981224 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.980662    6116 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 01:35:14.981224 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.980680    6116 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 01:35:14.981224 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.980725    6116 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 01:29:03 +0000 UTC" deadline="2028-07-09 00:17:39.610467846 +0000 UTC"
Nov 04 01:35:14.981224 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.980759    6116 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14710h42m24.629712581s"
Nov 04 01:35:14.981224 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.980790    6116 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 01:35:14.981224 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.980797    6116 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 01:35:14.981224 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:14.980853    6116 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-138.us-west-1.compute.internal\" not found"
Nov 04 01:35:14.981224 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.980870    6116 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 01:35:14.982270 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.982244    6116 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 01:35:14.983704 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983665    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce" volumeName="kubernetes.io/secret/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-insights-runtime-extractor-tls" seLinuxMountContext=""
Nov 04 01:35:14.983704 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983702    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-1" seLinuxMountContext=""
Nov 04 01:35:14.983704 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983711    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-trusted-ca-bundle" seLinuxMountContext=""
Nov 04 01:35:14.983704 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983718    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-db" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983725    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-config" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983732    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e62bee01-e717-4484-bb3d-ff4af522a355" volumeName="kubernetes.io/projected/e62bee01-e717-4484-bb3d-ff4af522a355-kube-api-access-4gmb7" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983739    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="37d34756-13c3-4993-a3d1-07f68eada6b0" volumeName="kubernetes.io/empty-dir/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-tuned" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983746    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52cfdb11-898a-4673-97c5-ecddc1385a03" volumeName="kubernetes.io/configmap/52cfdb11-898a-4673-97c5-ecddc1385a03-iptables-alerter-script" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983755    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-serving-certs-ca-bundle" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983762    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-tls" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983769    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="010e4cd1-587e-43a7-9f58-93ca75a074df" volumeName="kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-oauth-serving-cert" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983775    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4044e7b0-63ac-4b5f-af3b-5845b75f7c6f" volumeName="kubernetes.io/configmap/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-config-volume" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983782    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4044e7b0-63ac-4b5f-af3b-5845b75f7c6f" volumeName="kubernetes.io/projected/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-kube-api-access-4fqmn" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983789    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4044e7b0-63ac-4b5f-af3b-5845b75f7c6f" volumeName="kubernetes.io/secret/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-metrics-tls" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983811    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="469986a0-2b14-47a4-8f4d-3a1456447cdb" volumeName="kubernetes.io/configmap/469986a0-2b14-47a4-8f4d-3a1456447cdb-metrics-client-ca" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983818    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8f004fb-8c9c-4efb-b60d-358e32ba5120" volumeName="kubernetes.io/configmap/a8f004fb-8c9c-4efb-b60d-358e32ba5120-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983828    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-2" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983836    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-0" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983842    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="469986a0-2b14-47a4-8f4d-3a1456447cdb" volumeName="kubernetes.io/empty-dir/469986a0-2b14-47a4-8f4d-3a1456447cdb-volume-directive-shadow" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983850    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="67ed6df2-450d-4705-8a11-f98d5b507dc4" volumeName="kubernetes.io/configmap/67ed6df2-450d-4705-8a11-f98d5b507dc4-cni-binary-copy" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983867    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8f004fb-8c9c-4efb-b60d-358e32ba5120" volumeName="kubernetes.io/configmap/a8f004fb-8c9c-4efb-b60d-358e32ba5120-cni-binary-copy" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983878    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-kube-rbac-proxy" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983900    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983906    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e62bee01-e717-4484-bb3d-ff4af522a355" volumeName="kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983913    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce" volumeName="kubernetes.io/configmap/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-kube-rbac-proxy-cm" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983924    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="010e4cd1-587e-43a7-9f58-93ca75a074df" volumeName="kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-service-ca" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983936    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="010e4cd1-587e-43a7-9f58-93ca75a074df" volumeName="kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-console-config" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983943    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="192e2fec-fefb-4bc5-9516-ed7464963a10" volumeName="kubernetes.io/configmap/192e2fec-fefb-4bc5-9516-ed7464963a10-ovnkube-script-lib" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983955    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="192e2fec-fefb-4bc5-9516-ed7464963a10" volumeName="kubernetes.io/projected/192e2fec-fefb-4bc5-9516-ed7464963a10-kube-api-access-gxtbz" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983963    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa398d49-e1ee-41df-8785-b765e1f35b18" volumeName="kubernetes.io/projected/aa398d49-e1ee-41df-8785-b765e1f35b18-kube-api-access-lzftg" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983970    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb15eb62-434d-4cdd-a2dd-d1e069bb6abb" volumeName="kubernetes.io/configmap/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-metrics-client-ca" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983979    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="67ed6df2-450d-4705-8a11-f98d5b507dc4" volumeName="kubernetes.io/configmap/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-daemon-config" seLinuxMountContext=""
Nov 04 01:35:14.983967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983986    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8f004fb-8c9c-4efb-b60d-358e32ba5120" volumeName="kubernetes.io/configmap/a8f004fb-8c9c-4efb-b60d-358e32ba5120-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.983994    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce" volumeName="kubernetes.io/projected/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-kube-api-access-lls22" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984002    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="010e4cd1-587e-43a7-9f58-93ca75a074df" volumeName="kubernetes.io/empty-dir/010e4cd1-587e-43a7-9f58-93ca75a074df-tmp" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984008    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="010e4cd1-587e-43a7-9f58-93ca75a074df" volumeName="kubernetes.io/projected/010e4cd1-587e-43a7-9f58-93ca75a074df-kube-api-access-4nrq5" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984015    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20296e8e-e25f-40e1-9b2b-4971b7e69141" volumeName="kubernetes.io/projected/20296e8e-e25f-40e1-9b2b-4971b7e69141-kube-api-access-58s5t" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984023    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52cfdb11-898a-4673-97c5-ecddc1385a03" volumeName="kubernetes.io/projected/52cfdb11-898a-4673-97c5-ecddc1385a03-kube-api-access-m6fjs" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984030    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7acf7f6e-efb8-498e-8066-a60fbedecd49" volumeName="kubernetes.io/projected/7acf7f6e-efb8-498e-8066-a60fbedecd49-kube-api-access-4vz9t" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984037    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984044    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-config-out" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984051    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-metrics-client-certs" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984058    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="010e4cd1-587e-43a7-9f58-93ca75a074df" volumeName="kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-serving-cert" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984066    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="67ed6df2-450d-4705-8a11-f98d5b507dc4" volumeName="kubernetes.io/projected/67ed6df2-450d-4705-8a11-f98d5b507dc4-kube-api-access-gxtjk" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984072    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-grpc-tls" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984079    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb15eb62-434d-4cdd-a2dd-d1e069bb6abb" volumeName="kubernetes.io/projected/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-kube-api-access-dq7r6" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984085    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb15eb62-434d-4cdd-a2dd-d1e069bb6abb" volumeName="kubernetes.io/secret/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984093    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc" volumeName="kubernetes.io/secret/0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc-monitoring-plugin-cert" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984099    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f" volumeName="kubernetes.io/secret/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984106    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f" volumeName="kubernetes.io/secret/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-tls" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984113    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="010e4cd1-587e-43a7-9f58-93ca75a074df" volumeName="kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-trusted-ca-bundle" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984121    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="010e4cd1-587e-43a7-9f58-93ca75a074df" volumeName="kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-oauth-config" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984128    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="192e2fec-fefb-4bc5-9516-ed7464963a10" volumeName="kubernetes.io/configmap/192e2fec-fefb-4bc5-9516-ed7464963a10-ovnkube-config" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984135    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="192e2fec-fefb-4bc5-9516-ed7464963a10" volumeName="kubernetes.io/secret/192e2fec-fefb-4bc5-9516-ed7464963a10-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984141    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="480e250d-44a3-4d5c-987d-22737107544e" volumeName="kubernetes.io/secret/480e250d-44a3-4d5c-987d-22737107544e-agent-certs" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984148    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f" volumeName="kubernetes.io/configmap/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-metrics-client-ca" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984175    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-metrics-client-ca" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984182    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="469986a0-2b14-47a4-8f4d-3a1456447cdb" volumeName="kubernetes.io/projected/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-api-access-4nrqq" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984189    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce" volumeName="kubernetes.io/empty-dir/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-data-volume" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984196    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984204    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc" volumeName="kubernetes.io/projected/0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc-kube-api-access-28b8j" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984212    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="37d34756-13c3-4993-a3d1-07f68eada6b0" volumeName="kubernetes.io/projected/37d34756-13c3-4993-a3d1-07f68eada6b0-kube-api-access-xvlmf" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984219    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="469986a0-2b14-47a4-8f4d-3a1456447cdb" volumeName="kubernetes.io/secret/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984226    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f" volumeName="kubernetes.io/empty-dir/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-textfile" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984234    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ca08235e-943c-4d8d-939d-4ceaedd9cf92" volumeName="kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984240    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb15eb62-434d-4cdd-a2dd-d1e069bb6abb" volumeName="kubernetes.io/secret/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-prometheus-operator-tls" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984247    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20296e8e-e25f-40e1-9b2b-4971b7e69141" volumeName="kubernetes.io/configmap/20296e8e-e25f-40e1-9b2b-4971b7e69141-serviceca" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984255    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4044e7b0-63ac-4b5f-af3b-5845b75f7c6f" volumeName="kubernetes.io/empty-dir/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-tmp-dir" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984263    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="480e250d-44a3-4d5c-987d-22737107544e" volumeName="kubernetes.io/configmap/480e250d-44a3-4d5c-987d-22737107544e-konnectivity-ca" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984270    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8f004fb-8c9c-4efb-b60d-358e32ba5120" volumeName="kubernetes.io/projected/a8f004fb-8c9c-4efb-b60d-358e32ba5120-kube-api-access-7647t" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984277    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa398d49-e1ee-41df-8785-b765e1f35b18" volumeName="kubernetes.io/empty-dir/aa398d49-e1ee-41df-8785-b765e1f35b18-tmp-dir" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984284    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-thanos-prometheus-http-client-file" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984291    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="134fc8ea-90c5-4b4c-86a3-7b83d950f3e5" volumeName="kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984298    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e5c731f-7d75-42d6-8ebd-ee742632fa3d" volumeName="kubernetes.io/projected/1e5c731f-7d75-42d6-8ebd-ee742632fa3d-kube-api-access-5fjld" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984305    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="469986a0-2b14-47a4-8f4d-3a1456447cdb" volumeName="kubernetes.io/configmap/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984312    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="469986a0-2b14-47a4-8f4d-3a1456447cdb" volumeName="kubernetes.io/secret/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-state-metrics-tls" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984319    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f" volumeName="kubernetes.io/projected/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-kube-api-access-lrqc6" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984326    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-tls-assets" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984333    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-kube-api-access-v72sb" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984341    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cfd725f9-4be0-4be3-bb6a-898213f63761" volumeName="kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-web-config" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984349    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="192e2fec-fefb-4bc5-9516-ed7464963a10" volumeName="kubernetes.io/configmap/192e2fec-fefb-4bc5-9516-ed7464963a10-env-overrides" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984356    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e5c731f-7d75-42d6-8ebd-ee742632fa3d" volumeName="kubernetes.io/secret/1e5c731f-7d75-42d6-8ebd-ee742632fa3d-cert" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984363    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f" volumeName="kubernetes.io/configmap/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984379    6116 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="37d34756-13c3-4993-a3d1-07f68eada6b0" volumeName="kubernetes.io/empty-dir/37d34756-13c3-4993-a3d1-07f68eada6b0-tmp" seLinuxMountContext=""
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984387    6116 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 01:35:14.984824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.984392    6116 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 01:35:14.986549 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.985039    6116 factory.go:55] Registering systemd factory
Nov 04 01:35:14.986549 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.985054    6116 factory.go:223] Registration of the systemd container factory successfully
Nov 04 01:35:14.986549 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.985295    6116 factory.go:153] Registering CRI-O factory
Nov 04 01:35:14.986549 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.985305    6116 factory.go:223] Registration of the crio container factory successfully
Nov 04 01:35:14.986549 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.985368    6116 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 01:35:14.986549 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.985386    6116 factory.go:103] Registering Raw factory
Nov 04 01:35:14.986549 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.985399    6116 manager.go:1199] Started watching for new ooms in manager
Nov 04 01:35:14.986549 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:14.985738    6116 manager.go:319] Starting recovery of all containers
Nov 04 01:35:15.017956 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.017862    6116 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 01:35:15.018959 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.018936    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6lfbv_192e2fec-fefb-4bc5-9516-ed7464963a10/ovn-acl-logging/0.log"
Nov 04 01:35:15.019086 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.019031    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6lfbv_192e2fec-fefb-4bc5-9516-ed7464963a10/ovn-acl-logging/0.log"
Nov 04 01:35:15.019942 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.019912    6116 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 01:35:15.019942 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.019939    6116 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 01:35:15.020087 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.020013    6116 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 01:35:15.020133 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:15.020078    6116 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 04 01:35:15.045764 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.045729    6116 generic.go:356] "Generic (PLEG): container finished" podID="a8f004fb-8c9c-4efb-b60d-358e32ba5120" containerID="e42bb43a71c16cce015d035a2457940e5a8fa26844c07625808923eae11153d9" exitCode=0
Nov 04 01:35:15.045764 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.045760    6116 generic.go:356] "Generic (PLEG): container finished" podID="a8f004fb-8c9c-4efb-b60d-358e32ba5120" containerID="db52c6b2e59ed417d5e9f3b78599a8c83870ae017e010039390bac5a8766ce3e" exitCode=0
Nov 04 01:35:15.045764 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.045766    6116 generic.go:356] "Generic (PLEG): container finished" podID="a8f004fb-8c9c-4efb-b60d-358e32ba5120" containerID="cfbd049c9dfc4f539f39f9bc4b86591d4a16d2123c9f9e50c8d48d1f0778386b" exitCode=0
Nov 04 01:35:15.045764 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.045772    6116 generic.go:356] "Generic (PLEG): container finished" podID="a8f004fb-8c9c-4efb-b60d-358e32ba5120" containerID="ebc9ff9a537320587ec7c17c2342bc8ef2f02c8d8be9c51f7b028a42050df65e" exitCode=0
Nov 04 01:35:15.046082 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.045777    6116 generic.go:356] "Generic (PLEG): container finished" podID="a8f004fb-8c9c-4efb-b60d-358e32ba5120" containerID="b8a071b045d441da7f8a5c7fca9e9ede664e33494109085586489e7500ce7c6d" exitCode=0
Nov 04 01:35:15.046082 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.045782    6116 generic.go:356] "Generic (PLEG): container finished" podID="a8f004fb-8c9c-4efb-b60d-358e32ba5120" containerID="a451e1ac3a8bbb90ce38288c508334767da67aa3b32bd20f28f505988eb2372a" exitCode=0
Nov 04 01:35:15.057409 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.057373    6116 generic.go:356] "Generic (PLEG): container finished" podID="4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f" containerID="0f0be9a1950d9679b2d729820067d1de5dc56227a3e97856f86727dc22f92372" exitCode=0
Nov 04 01:35:15.074328 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.074301    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6lfbv_192e2fec-fefb-4bc5-9516-ed7464963a10/ovn-acl-logging/0.log"
Nov 04 01:35:15.074789 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.074755    6116 generic.go:356] "Generic (PLEG): container finished" podID="192e2fec-fefb-4bc5-9516-ed7464963a10" containerID="ac8621fa6abe76d4d71b32912b440560bb522bdf5eb5471572fbc94ca4518f2a" exitCode=1
Nov 04 01:35:15.081005 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:15.080983    6116 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-138.us-west-1.compute.internal\" not found"
Nov 04 01:35:15.090833 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.090809    6116 manager.go:324] Recovery completed
Nov 04 01:35:15.094035 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.093665    6116 generic.go:356] "Generic (PLEG): container finished" podID="c58b01dca1916bc539c28fce5659b181" containerID="bf475d1803d2ac5df8f1e3bbca0232a0e67e021a6ddb6fa85997dddb2ade8527" exitCode=0
Nov 04 01:35:15.099723 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.099693    6116 generic.go:356] "Generic (PLEG): container finished" podID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerID="a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7" exitCode=0
Nov 04 01:35:15.111922 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.111893    6116 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:35:15.114149 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.114126    6116 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:35:15.114268 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.114166    6116 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:35:15.114268 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.114181    6116 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:35:15.116820 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.116800    6116 cpu_manager.go:226] "Starting" policy="none"
Nov 04 01:35:15.116820 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.116816    6116 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 01:35:15.116964 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.116837    6116 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 01:35:15.117013 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.116991    6116 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 04 01:35:15.117071 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.117006    6116 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 04 01:35:15.117117 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.117107    6116 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 04 01:35:15.117167 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.117117    6116 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 04 01:35:15.117167 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.117125    6116 policy_none.go:50] "Start"
Nov 04 01:35:15.117167 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.117138    6116 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 01:35:15.117167 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.117151    6116 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 01:35:15.117330 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.117210    6116 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:35:15.117376 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.117334    6116 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 04 01:35:15.117376 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.117345    6116 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 04 01:35:15.117376 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.117354    6116 policy_none.go:44] "Start"
Nov 04 01:35:15.118862 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.118835    6116 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:35:15.120661 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:15.120635    6116 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Nov 04 01:35:15.121422 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.121400    6116 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:35:15.121533 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.121439    6116 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:35:15.121533 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.121477    6116 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:35:15.124570 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.124536    6116 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 01:35:15.129409 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.129390    6116 manager.go:341] "Starting Device Plugin manager"
Nov 04 01:35:15.129524 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:15.129425    6116 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 01:35:15.129524 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.129438    6116 server.go:87] "Starting device plugin registration server"
Nov 04 01:35:15.129791 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.129778    6116 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 01:35:15.130436 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.129792    6116 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 01:35:15.130544 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.130533    6116 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 01:35:15.130671 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.130636    6116 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 01:35:15.130722 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.130679    6116 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 01:35:15.130722 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.130685    6116 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 01:35:15.130810 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.130737    6116 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T01:35:15.130669104Z","UUID":"522e5190-d57d-414d-bca7-d6a34a9c6eff","Handler":null,"Name":"","Endpoint":""}
Nov 04 01:35:15.132009 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:15.131961    6116 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 01:35:15.132098 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:15.132035    6116 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-138.us-west-1.compute.internal\" not found"
Nov 04 01:35:15.132256 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.132242    6116 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 01:35:15.132347 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.132275    6116 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 01:35:15.167773 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.167739    6116 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:35:15.167994 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.167910    6116 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:35:15.231156 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.231111    6116 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 01:35:15.232348 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.232329    6116 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 01:35:15.232498 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.232358    6116 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 01:35:15.232498 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.232368    6116 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-138.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 01:35:15.232498 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.232392    6116 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:15.265929 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.265883    6116 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 04 01:35:15.265929 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.265898    6116 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 01:35:15.321269 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.321157    6116 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal"]
Nov 04 01:35:15.322046 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.322024    6116 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:15.322046 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.322024    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal" event={"ID":"5d9a5980aefdf4e226d40970221f0eb6","Type":"ContainerStarted","Data":"7ecf8fc37ef1fc332541c023e68a53059b0f5c2b9c1bb2742442b65ef023d0f5"}
Nov 04 01:35:15.322233 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.322071    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal" event={"ID":"5d9a5980aefdf4e226d40970221f0eb6","Type":"ContainerStarted","Data":"5471b6fa46f5222a9d35b493eaece92ee03c1f1d0b93043685e526dfe5784179"}
Nov 04 01:35:15.322233 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.322129    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal" event={"ID":"c58b01dca1916bc539c28fce5659b181","Type":"ContainerStarted","Data":"9cd1628279a72a1a5bdad88e54a4451d55f29753bb8a2ddc91d63b3d155281ec"}
Nov 04 01:35:15.322233 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.322151    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal" event={"ID":"c58b01dca1916bc539c28fce5659b181","Type":"ContainerDied","Data":"bf475d1803d2ac5df8f1e3bbca0232a0e67e021a6ddb6fa85997dddb2ade8527"}
Nov 04 01:35:15.322233 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.322170    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal" event={"ID":"c58b01dca1916bc539c28fce5659b181","Type":"ContainerStarted","Data":"84fb8fd6e9424f9f7ce12f856c19373e6bee5a388fedaa3eabd1636fa96c6389"}
Nov 04 01:35:15.322392 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.322336    6116 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:15.341227 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.341194    6116 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:15.341392 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.341275    6116 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:15.382798 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.382763    6116 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:35:15.382990 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:15.382813    6116 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:15.383311 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.383292    6116 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:35:15.383421 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:15.383336    6116 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:15.466554 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.466505    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c58b01dca1916bc539c28fce5659b181-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal\" (UID: \"c58b01dca1916bc539c28fce5659b181\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:15.466731 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.466560    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5d9a5980aefdf4e226d40970221f0eb6-config\") pod \"kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal\" (UID: \"5d9a5980aefdf4e226d40970221f0eb6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:15.466731 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.466631    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c58b01dca1916bc539c28fce5659b181-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal\" (UID: \"c58b01dca1916bc539c28fce5659b181\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:15.567239 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.567200    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5d9a5980aefdf4e226d40970221f0eb6-config\") pod \"kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal\" (UID: \"5d9a5980aefdf4e226d40970221f0eb6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:15.567239 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.567239    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c58b01dca1916bc539c28fce5659b181-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal\" (UID: \"c58b01dca1916bc539c28fce5659b181\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:15.567464 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.567255    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c58b01dca1916bc539c28fce5659b181-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal\" (UID: \"c58b01dca1916bc539c28fce5659b181\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:15.567464 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.567302    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c58b01dca1916bc539c28fce5659b181-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal\" (UID: \"c58b01dca1916bc539c28fce5659b181\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:15.567464 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.567361    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5d9a5980aefdf4e226d40970221f0eb6-config\") pod \"kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal\" (UID: \"5d9a5980aefdf4e226d40970221f0eb6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:15.567464 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.567397    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c58b01dca1916bc539c28fce5659b181-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal\" (UID: \"c58b01dca1916bc539c28fce5659b181\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:15.956732 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:15.956694    6116 apiserver.go:52] "Watching apiserver"
Nov 04 01:35:16.060162 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.060121    6116 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 01:35:16.062162 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.061861    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-hh4tb","kube-system/konnectivity-agent-kwbdd","openshift-dns/dns-default-76b68","kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx","openshift-dns/node-resolver-zpmgq","openshift-console/console-6cdfff847d-84rl8","openshift-ingress-canary/ingress-canary-7lrcs","openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch","openshift-ovn-kubernetes/ovnkube-node-6lfbv","openshift-multus/network-metrics-daemon-q7q6c","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal","kube-system/global-pull-secret-syncer-hjqq9","openshift-multus/multus-hcvld","openshift-network-diagnostics/network-check-target-hbs2c","openshift-network-operator/iptables-alerter-2sbtc","openshift-insights/insights-runtime-extractor-8zjpf","openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd","openshift-cluster-node-tuning-operator/tuned-hhttg","openshift-image-registry/node-ca-z992q","openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl","openshift-monitoring/node-exporter-27c6h","openshift-monitoring/prometheus-k8s-0"]
Nov 04 01:35:16.082743 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.082717    6116 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 01:35:16.105779 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.105748    6116 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:16.105925 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.105849    6116 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:16.122582 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.122542    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 01:35:16.122842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.122582    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 01:35:16.122842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.122612    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 01:35:16.122842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.122659    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 01:35:16.122842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.122664    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vr5s4\""
Nov 04 01:35:16.122842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.122695    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:35:16.122842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.122747    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 01:35:16.122842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.122758    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 01:35:16.122842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.122853    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 01:35:16.122842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.122855    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mh5l6\""
Nov 04 01:35:16.122842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.122871    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 01:35:16.123358 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.122967    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 01:35:16.123358 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.122997    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 01:35:16.123358 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.123096    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 01:35:16.123358 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.123101    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 01:35:16.123358 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.123254    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 01:35:16.123358 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.123268    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 01:35:16.123753 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.123735    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 01:35:16.123753 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.123748    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 01:35:16.123913 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.123760    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 01:35:16.123913 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.123779    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 01:35:16.123913 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.123790    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 01:35:16.123913 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.123821    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 01:35:16.123913 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.123828    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 01:35:16.123913 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.123827    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 01:35:16.123913 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.123894    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 01:35:16.123913 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.123916    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:35:16.124252 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.124239    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 01:35:16.124289 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.124253    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 01:35:16.124289 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.124275    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 01:35:16.124354 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.124288    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 01:35:16.124354 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.124301    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lhd4j\""
Nov 04 01:35:16.133267 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.133247    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 01:35:16.135083 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.135062    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 01:35:16.135890 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.135860    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 01:35:16.136113 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.136093    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-shfxh\""
Nov 04 01:35:16.136232 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.136205    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 01:35:16.136299 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.136253    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xkhwh\""
Nov 04 01:35:16.136299 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.136275    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xwn9r\""
Nov 04 01:35:16.136299 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.136284    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-gnb5c\""
Nov 04 01:35:16.136299 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.136285    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-vqsf7\""
Nov 04 01:35:16.136487 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.136351    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-ldpmc\""
Nov 04 01:35:16.136487 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.136396    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 01:35:16.136487 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.136401    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ctj2f\""
Nov 04 01:35:16.136487 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.136412    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tfqdd\""
Nov 04 01:35:16.136487 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.136414    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5jndn\""
Nov 04 01:35:16.136487 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.136415    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zbbgd\""
Nov 04 01:35:16.136720 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.136503    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-sj5vk\""
Nov 04 01:35:16.136720 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.136637    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 01:35:16.164404 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.164373    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 04 01:35:16.164544 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.164440    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 01:35:16.164613 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.164593    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 01:35:16.164856 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.164831    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Nov 04 01:35:16.164979 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.164865    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1eqn9lagl0d1b\""
Nov 04 01:35:16.164979 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.164865    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Nov 04 01:35:16.164979 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.164886    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Nov 04 01:35:16.164979 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.164929    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Nov 04 01:35:16.165202 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.165147    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bnxvr\""
Nov 04 01:35:16.165202 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.165176    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 04 01:35:16.165429 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.165415    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Nov 04 01:35:16.165582 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.165570    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Nov 04 01:35:16.165660 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.165592    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Nov 04 01:35:16.165713 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.165698    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 04 01:35:16.165935 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.165920    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 01:35:16.166027 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.165942    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Nov 04 01:35:16.166027 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.165973    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Nov 04 01:35:16.166027 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.165975    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-nbgcc\""
Nov 04 01:35:16.166027 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.166002    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 04 01:35:16.166027 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.166004    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Nov 04 01:35:16.166301 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.166250    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 04 01:35:16.166301 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.166252    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 04 01:35:16.166301 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.166270    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 04 01:35:16.166301 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.166272    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Nov 04 01:35:16.166301 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.166272    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2jjdl\""
Nov 04 01:35:16.166301 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.166302    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 04 01:35:16.166560 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.166287    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 04 01:35:16.166560 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.166255    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 04 01:35:16.166560 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.166375    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Nov 04 01:35:16.166560 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.166395    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Nov 04 01:35:16.166560 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.166399    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 04 01:35:16.169866 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.169846    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 01:35:16.169866 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.169846    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 01:35:16.169979 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.169875    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j6ln8\""
Nov 04 01:35:16.169979 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.169851    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 01:35:16.169979 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.169846    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2bmnb\""
Nov 04 01:35:16.169979 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.169876    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Nov 04 01:35:16.170733 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.170689    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-9mnjb\""
Nov 04 01:35:16.170821 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.170792    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 01:35:16.173084 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173066    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-g7lb4\""
Nov 04 01:35:16.173199 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173092    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Nov 04 01:35:16.173308 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173104    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Nov 04 01:35:16.173308 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173152    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Nov 04 01:35:16.173462 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173192    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/192e2fec-fefb-4bc5-9516-ed7464963a10-ovn-node-metrics-cert\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.173462 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173217    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Nov 04 01:35:16.173462 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173235    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 04 01:35:16.173462 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173349    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-modprobe-d\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.173462 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173421    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52cfdb11-898a-4673-97c5-ecddc1385a03-host-slash\") pod \"iptables-alerter-2sbtc\" (UID: \"52cfdb11-898a-4673-97c5-ecddc1385a03\") " pod="openshift-network-operator/iptables-alerter-2sbtc"
Nov 04 01:35:16.173462 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173461    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8f004fb-8c9c-4efb-b60d-358e32ba5120-cnibin\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:35:16.173700 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173512    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxtbz\" (UniqueName: \"kubernetes.io/projected/192e2fec-fefb-4bc5-9516-ed7464963a10-kube-api-access-gxtbz\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.173700 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173580    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/192e2fec-fefb-4bc5-9516-ed7464963a10-ovn-node-metrics-cert\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.173700 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173598    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-tuned\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.173700 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173624    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7647t\" (UniqueName: \"kubernetes.io/projected/a8f004fb-8c9c-4efb-b60d-358e32ba5120-kube-api-access-7647t\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:35:16.173700 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173649    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-service-ca\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:16.173700 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173666    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-tls\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.173700 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173681    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-tuned\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.173998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173713    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lls22\" (UniqueName: \"kubernetes.io/projected/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-kube-api-access-lls22\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:35:16.173998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173736    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-var-lib-cni-multus\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.173998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173750    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-run-multus-certs\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.173998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173770    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-etc-selinux\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.173998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173793    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-oauth-config\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:16.173998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173838    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gmb7\" (UniqueName: \"kubernetes.io/projected/e62bee01-e717-4484-bb3d-ff4af522a355-kube-api-access-4gmb7\") pod \"network-metrics-daemon-q7q6c\" (UID: \"e62bee01-e717-4484-bb3d-ff4af522a355\") " pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:35:16.173998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173862    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/469986a0-2b14-47a4-8f4d-3a1456447cdb-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:35:16.173998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173880    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-service-ca\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:16.173998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173885    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-run-k8s-cni-cncf-io\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.173998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173914    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.173998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173941    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-console-config\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:16.173998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173972    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-var-lib-openvswitch\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.173998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.173996    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/192e2fec-fefb-4bc5-9516-ed7464963a10-ovnkube-config\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.174652 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174012    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/20296e8e-e25f-40e1-9b2b-4971b7e69141-serviceca\") pod \"node-ca-z992q\" (UID: \"20296e8e-e25f-40e1-9b2b-4971b7e69141\") " pod="openshift-image-registry/node-ca-z992q"
Nov 04 01:35:16.174652 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174031    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-sys\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.174652 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174051    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.174652 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174065    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-run-openvswitch\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.174652 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174125    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-tls\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.174652 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174135    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58s5t\" (UniqueName: \"kubernetes.io/projected/20296e8e-e25f-40e1-9b2b-4971b7e69141-kube-api-access-58s5t\") pod \"node-ca-z992q\" (UID: \"20296e8e-e25f-40e1-9b2b-4971b7e69141\") " pod="openshift-image-registry/node-ca-z992q"
Nov 04 01:35:16.174652 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174203    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-lib-modules\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.174652 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174251    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvlmf\" (UniqueName: \"kubernetes.io/projected/37d34756-13c3-4993-a3d1-07f68eada6b0-kube-api-access-xvlmf\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.174652 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174252    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-console-config\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:16.174652 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174281    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:35:16.174652 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174317    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:35:16.174652 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174364    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-oauth-config\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:16.174652 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174500    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq7r6\" (UniqueName: \"kubernetes.io/projected/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-kube-api-access-dq7r6\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:35:16.174652 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174506    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.174652 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174563    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/469986a0-2b14-47a4-8f4d-3a1456447cdb-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:35:16.174652 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174569    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.174652 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174643    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8f004fb-8c9c-4efb-b60d-358e32ba5120-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:35:16.175174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174692    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:35:16.175174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174692    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:35:16.175174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174717    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/192e2fec-fefb-4bc5-9516-ed7464963a10-ovnkube-config\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.175174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174747    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/010e4cd1-587e-43a7-9f58-93ca75a074df-tmp\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:16.175174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174807    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/010e4cd1-587e-43a7-9f58-93ca75a074df-tmp\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:16.175174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174813    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.175174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174829    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-w4mgl\" (UID: \"0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"
Nov 04 01:35:16.175174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174848    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.175174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174878    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-dbus\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:35:16.175174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.174973    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-metrics-tls\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:35:16.175174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175029    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.175174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175067    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-etc-kubernetes\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.175174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175075    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/20296e8e-e25f-40e1-9b2b-4971b7e69141-serviceca\") pod \"node-ca-z992q\" (UID: \"20296e8e-e25f-40e1-9b2b-4971b7e69141\") " pod="openshift-image-registry/node-ca-z992q"
Nov 04 01:35:16.175174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175133    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a8f004fb-8c9c-4efb-b60d-358e32ba5120-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:35:16.175174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175140    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-w4mgl\" (UID: \"0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"
Nov 04 01:35:16.175174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175172    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-var-lib-kubelet\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.175612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175225    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-metrics-tls\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:35:16.175612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175276    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxtjk\" (UniqueName: \"kubernetes.io/projected/67ed6df2-450d-4705-8a11-f98d5b507dc4-kube-api-access-gxtjk\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.175612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175304    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-sys-fs\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.175612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175317    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a8f004fb-8c9c-4efb-b60d-358e32ba5120-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:35:16.175612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175326    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-log-socket\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.175612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175353    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-wtmp\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.175612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175389    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-tmp-dir\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:35:16.175612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175421    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:35:16.175612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175468    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-tmp-dir\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:35:16.175612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175470    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-var-lib-cni-bin\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.175612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175489    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-socket-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.175612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175506    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nrq5\" (UniqueName: \"kubernetes.io/projected/010e4cd1-587e-43a7-9f58-93ca75a074df-kube-api-access-4nrq5\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:16.175612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175524    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-cni-netd\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.175612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175546    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-var-lib-kubelet\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.175612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175595    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:35:16.175989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175628    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-host\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.175989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175649    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:35:16.175989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175687    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.175989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175713    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/480e250d-44a3-4d5c-987d-22737107544e-konnectivity-ca\") pod \"konnectivity-agent-kwbdd\" (UID: \"480e250d-44a3-4d5c-987d-22737107544e\") " pod="kube-system/konnectivity-agent-kwbdd"
Nov 04 01:35:16.175989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175765    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:35:16.175989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175813    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:35:16.175989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175858    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e5c731f-7d75-42d6-8ebd-ee742632fa3d-cert\") pod \"ingress-canary-7lrcs\" (UID: \"1e5c731f-7d75-42d6-8ebd-ee742632fa3d\") " pod="openshift-ingress-canary/ingress-canary-7lrcs"
Nov 04 01:35:16.175989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175866    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/480e250d-44a3-4d5c-987d-22737107544e-konnectivity-ca\") pod \"konnectivity-agent-kwbdd\" (UID: \"480e250d-44a3-4d5c-987d-22737107544e\") " pod="kube-system/konnectivity-agent-kwbdd"
Nov 04 01:35:16.175989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175884    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-root\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.175989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175931    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa398d49-e1ee-41df-8785-b765e1f35b18-tmp-dir\") pod \"node-resolver-zpmgq\" (UID: \"aa398d49-e1ee-41df-8785-b765e1f35b18\") " pod="openshift-dns/node-resolver-zpmgq"
Nov 04 01:35:16.175989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175968    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-system-cni-dir\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.176346 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.175992    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-hostroot\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.176346 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176019    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.176346 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176041    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa398d49-e1ee-41df-8785-b765e1f35b18-tmp-dir\") pod \"node-resolver-zpmgq\" (UID: \"aa398d49-e1ee-41df-8785-b765e1f35b18\") " pod="openshift-dns/node-resolver-zpmgq"
Nov 04 01:35:16.176346 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176049    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-node-log\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.176346 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176086    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e5c731f-7d75-42d6-8ebd-ee742632fa3d-cert\") pod \"ingress-canary-7lrcs\" (UID: \"1e5c731f-7d75-42d6-8ebd-ee742632fa3d\") " pod="openshift-ingress-canary/ingress-canary-7lrcs"
Nov 04 01:35:16.176346 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176088    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-original-pull-secret\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:35:16.176346 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176109    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-kubelet-config\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:35:16.176346 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176137    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aa398d49-e1ee-41df-8785-b765e1f35b18-hosts-file\") pod \"node-resolver-zpmgq\" (UID: \"aa398d49-e1ee-41df-8785-b765e1f35b18\") " pod="openshift-dns/node-resolver-zpmgq"
Nov 04 01:35:16.176346 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176164    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6fjs\" (UniqueName: \"kubernetes.io/projected/52cfdb11-898a-4673-97c5-ecddc1385a03-kube-api-access-m6fjs\") pod \"iptables-alerter-2sbtc\" (UID: \"52cfdb11-898a-4673-97c5-ecddc1385a03\") " pod="openshift-network-operator/iptables-alerter-2sbtc"
Nov 04 01:35:16.176346 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176249    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.176346 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176301    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a8f004fb-8c9c-4efb-b60d-358e32ba5120-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:35:16.176346 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176332    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-oauth-serving-cert\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176357    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-slash\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176382    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fjld\" (UniqueName: \"kubernetes.io/projected/1e5c731f-7d75-42d6-8ebd-ee742632fa3d-kube-api-access-5fjld\") pod \"ingress-canary-7lrcs\" (UID: \"1e5c731f-7d75-42d6-8ebd-ee742632fa3d\") " pod="openshift-ingress-canary/ingress-canary-7lrcs"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176408    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37d34756-13c3-4993-a3d1-07f68eada6b0-tmp\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176467    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176474    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176511    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176482    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a8f004fb-8c9c-4efb-b60d-358e32ba5120-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176542    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37d34756-13c3-4993-a3d1-07f68eada6b0-tmp\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176549    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-oauth-serving-cert\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176551    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v72sb\" (UniqueName: \"kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-kube-api-access-v72sb\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176589    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-etc-openvswitch\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176615    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/192e2fec-fefb-4bc5-9516-ed7464963a10-env-overrides\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176645    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176685    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176713    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nrqq\" (UniqueName: \"kubernetes.io/projected/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-api-access-4nrqq\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176732    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-kubelet\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176742    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/192e2fec-fefb-4bc5-9516-ed7464963a10-env-overrides\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176824    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-config\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176842    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrqc6\" (UniqueName: \"kubernetes.io/projected/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-kube-api-access-lrqc6\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.176877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176857    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/469986a0-2b14-47a4-8f4d-3a1456447cdb-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176923    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/469986a0-2b14-47a4-8f4d-3a1456447cdb-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176970    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-web-config\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.176991    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-device-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177008    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177043    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-run-ovn\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177148    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-web-config\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177159    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-sys\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177191    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-sysconfig\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177229    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-config-out\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177255    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-config\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177266    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177293    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-data-volume\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177305    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-config-out\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177293    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177357    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-data-volume\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177379    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177396    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177429    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c8rj\" (UniqueName: \"kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj\") pod \"network-check-target-hbs2c\" (UID: \"ca08235e-943c-4d8d-939d-4ceaedd9cf92\") " pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177476    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177479    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177530    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-systemd-units\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177557    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:35:16.177574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177581    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177601    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177621    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-run\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177647    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177658    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/480e250d-44a3-4d5c-987d-22737107544e-agent-certs\") pod \"konnectivity-agent-kwbdd\" (UID: \"480e250d-44a3-4d5c-987d-22737107544e\") " pod="kube-system/konnectivity-agent-kwbdd"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177699    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/192e2fec-fefb-4bc5-9516-ed7464963a10-ovnkube-script-lib\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177721    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-textfile\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177737    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28b8j\" (UniqueName: \"kubernetes.io/projected/0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc-kube-api-access-28b8j\") pod \"monitoring-plugin-77f4f76594-w4mgl\" (UID: \"0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177763    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-systemd\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177780    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177788    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-socket-dir-parent\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177834    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-textfile\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177839    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vz9t\" (UniqueName: \"kubernetes.io/projected/7acf7f6e-efb8-498e-8066-a60fbedecd49-kube-api-access-4vz9t\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177882    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20296e8e-e25f-40e1-9b2b-4971b7e69141-host\") pod \"node-ca-z992q\" (UID: \"20296e8e-e25f-40e1-9b2b-4971b7e69141\") " pod="openshift-image-registry/node-ca-z992q"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177912    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/67ed6df2-450d-4705-8a11-f98d5b507dc4-cni-binary-copy\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.177939    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/192e2fec-fefb-4bc5-9516-ed7464963a10-ovnkube-script-lib\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.178033    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.178135    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-cni-bin\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.178151    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/67ed6df2-450d-4705-8a11-f98d5b507dc4-cni-binary-copy\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.178231    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzftg\" (UniqueName: \"kubernetes.io/projected/aa398d49-e1ee-41df-8785-b765e1f35b18-kube-api-access-lzftg\") pod \"node-resolver-zpmgq\" (UID: \"aa398d49-e1ee-41df-8785-b765e1f35b18\") " pod="openshift-dns/node-resolver-zpmgq"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.178270    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-sysctl-conf\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.178335    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-crio-socket\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:35:16.178344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.178346    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.178948 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.178365    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/52cfdb11-898a-4673-97c5-ecddc1385a03-iptables-alerter-script\") pod \"iptables-alerter-2sbtc\" (UID: \"52cfdb11-898a-4673-97c5-ecddc1385a03\") " pod="openshift-network-operator/iptables-alerter-2sbtc"
Nov 04 01:35:16.178948 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.178390    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/480e250d-44a3-4d5c-987d-22737107544e-agent-certs\") pod \"konnectivity-agent-kwbdd\" (UID: \"480e250d-44a3-4d5c-987d-22737107544e\") " pod="kube-system/konnectivity-agent-kwbdd"
Nov 04 01:35:16.178948 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.178404    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-conf-dir\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.178948 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.178487    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8f004fb-8c9c-4efb-b60d-358e32ba5120-system-cni-dir\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:35:16.178948 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.178521    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-run-systemd\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.178948 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.178562    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/52cfdb11-898a-4673-97c5-ecddc1385a03-iptables-alerter-script\") pod \"iptables-alerter-2sbtc\" (UID: \"52cfdb11-898a-4673-97c5-ecddc1385a03\") " pod="openshift-network-operator/iptables-alerter-2sbtc"
Nov 04 01:35:16.178948 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.178566    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-run-netns\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.178948 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.178592    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-run-ovn-kubernetes\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.178948 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.178830    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.179178 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179017    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.179178 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179044    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-run-netns\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.179178 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179059    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-registration-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.179178 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179076    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-trusted-ca-bundle\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:16.179178 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179090    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-cni-dir\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.179178 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179123    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.179178 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179153    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8f004fb-8c9c-4efb-b60d-358e32ba5120-cni-binary-copy\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:35:16.179418 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179186    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.179418 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179206    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.179418 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179225    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-metrics-client-ca\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.179418 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179234    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.179418 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179242    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs\") pod \"network-metrics-daemon-q7q6c\" (UID: \"e62bee01-e717-4484-bb3d-ff4af522a355\") " pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:35:16.179418 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179261    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-kubernetes\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.179418 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179388    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.179418 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179392    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-config-volume\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:35:16.179804 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179422    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8f004fb-8c9c-4efb-b60d-358e32ba5120-cni-binary-copy\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:35:16.179804 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179423    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-accelerators-collector-config\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.179804 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179472    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e62bee01-e717-4484-bb3d-ff4af522a355-metrics-certs\") pod \"network-metrics-daemon-q7q6c\" (UID: \"e62bee01-e717-4484-bb3d-ff4af522a355\") " pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:35:16.179804 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179507    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-sysctl-d\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.179804 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179544    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-daemon-config\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.179804 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179565    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-metrics-client-ca\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.179804 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179576    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-config-volume\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:35:16.179804 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179588    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.179804 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179611    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.179804 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179628    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:35:16.179804 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179657    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fqmn\" (UniqueName: \"kubernetes.io/projected/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-kube-api-access-4fqmn\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:35:16.179804 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179686    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-cnibin\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.179804 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179715    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-serving-cert\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:16.179804 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179634    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-accelerators-collector-config\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.179804 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179766    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-daemon-config\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.180242 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179828    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.180242 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179894    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.180242 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179910    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:35:16.180242 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.179958    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-serving-cert\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:16.214974 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.214868    6116 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:35:16.215132 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:16.214983    6116 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:16.215132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.215001    6116 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 01:35:16.215132 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:16.215095    6116 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal"
Nov 04 01:35:16.216713 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.215697    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Nov 04 01:35:16.216713 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.215945    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Nov 04 01:35:16.216713 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.216290    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Nov 04 01:35:16.216941 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.216720    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.219761 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.219735    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-trusted-ca-bundle\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:16.229341 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.229308    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.232879 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.232849    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7647t\" (UniqueName: \"kubernetes.io/projected/a8f004fb-8c9c-4efb-b60d-358e32ba5120-kube-api-access-7647t\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:35:16.233022 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.232849    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58s5t\" (UniqueName: \"kubernetes.io/projected/20296e8e-e25f-40e1-9b2b-4971b7e69141-kube-api-access-58s5t\") pod \"node-ca-z992q\" (UID: \"20296e8e-e25f-40e1-9b2b-4971b7e69141\") " pod="openshift-image-registry/node-ca-z992q"
Nov 04 01:35:16.233022 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.232861    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gmb7\" (UniqueName: \"kubernetes.io/projected/e62bee01-e717-4484-bb3d-ff4af522a355-kube-api-access-4gmb7\") pod \"network-metrics-daemon-q7q6c\" (UID: \"e62bee01-e717-4484-bb3d-ff4af522a355\") " pod="openshift-multus/network-metrics-daemon-q7q6c"
Nov 04 01:35:16.233022 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.232913    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvlmf\" (UniqueName: \"kubernetes.io/projected/37d34756-13c3-4993-a3d1-07f68eada6b0-kube-api-access-xvlmf\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.233190 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.233167    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq7r6\" (UniqueName: \"kubernetes.io/projected/eb15eb62-434d-4cdd-a2dd-d1e069bb6abb-kube-api-access-dq7r6\") pod \"prometheus-operator-9cd89b74c-mgxhd\" (UID: \"eb15eb62-434d-4cdd-a2dd-d1e069bb6abb\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-mgxhd"
Nov 04 01:35:16.233317 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.233296    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxtbz\" (UniqueName: \"kubernetes.io/projected/192e2fec-fefb-4bc5-9516-ed7464963a10-kube-api-access-gxtbz\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.233347 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.233296    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxtjk\" (UniqueName: \"kubernetes.io/projected/67ed6df2-450d-4705-8a11-f98d5b507dc4-kube-api-access-gxtjk\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.234201 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.234177    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lls22\" (UniqueName: \"kubernetes.io/projected/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-kube-api-access-lls22\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:35:16.235285 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.235260    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nrqq\" (UniqueName: \"kubernetes.io/projected/469986a0-2b14-47a4-8f4d-3a1456447cdb-kube-api-access-4nrqq\") pod \"kube-state-metrics-5dc8d7d89b-92fch\" (UID: \"469986a0-2b14-47a4-8f4d-3a1456447cdb\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-92fch"
Nov 04 01:35:16.235379 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.235350    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nrq5\" (UniqueName: \"kubernetes.io/projected/010e4cd1-587e-43a7-9f58-93ca75a074df-kube-api-access-4nrq5\") pod \"console-6cdfff847d-84rl8\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") " pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:16.235864 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.235842    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6fjs\" (UniqueName: \"kubernetes.io/projected/52cfdb11-898a-4673-97c5-ecddc1385a03-kube-api-access-m6fjs\") pod \"iptables-alerter-2sbtc\" (UID: \"52cfdb11-898a-4673-97c5-ecddc1385a03\") " pod="openshift-network-operator/iptables-alerter-2sbtc"
Nov 04 01:35:16.236068 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.236053    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v72sb\" (UniqueName: \"kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-kube-api-access-v72sb\") pod \"prometheus-k8s-0\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:16.236429 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.236414    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrqc6\" (UniqueName: \"kubernetes.io/projected/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-kube-api-access-lrqc6\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.238062 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.238038    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fjld\" (UniqueName: \"kubernetes.io/projected/1e5c731f-7d75-42d6-8ebd-ee742632fa3d-kube-api-access-5fjld\") pod \"ingress-canary-7lrcs\" (UID: \"1e5c731f-7d75-42d6-8ebd-ee742632fa3d\") " pod="openshift-ingress-canary/ingress-canary-7lrcs"
Nov 04 01:35:16.238197 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.238181    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c8rj\" (UniqueName: \"kubernetes.io/projected/ca08235e-943c-4d8d-939d-4ceaedd9cf92-kube-api-access-7c8rj\") pod \"network-check-target-hbs2c\" (UID: \"ca08235e-943c-4d8d-939d-4ceaedd9cf92\") " pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:35:16.238328 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.238310    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28b8j\" (UniqueName: \"kubernetes.io/projected/0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc-kube-api-access-28b8j\") pod \"monitoring-plugin-77f4f76594-w4mgl\" (UID: \"0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"
Nov 04 01:35:16.238463 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.238426    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fqmn\" (UniqueName: \"kubernetes.io/projected/4044e7b0-63ac-4b5f-af3b-5845b75f7c6f-kube-api-access-4fqmn\") pod \"dns-default-76b68\" (UID: \"4044e7b0-63ac-4b5f-af3b-5845b75f7c6f\") " pod="openshift-dns/dns-default-76b68"
Nov 04 01:35:16.238664 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.238649    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vz9t\" (UniqueName: \"kubernetes.io/projected/7acf7f6e-efb8-498e-8066-a60fbedecd49-kube-api-access-4vz9t\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.238799 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.238784    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzftg\" (UniqueName: \"kubernetes.io/projected/aa398d49-e1ee-41df-8785-b765e1f35b18-kube-api-access-lzftg\") pod \"node-resolver-zpmgq\" (UID: \"aa398d49-e1ee-41df-8785-b765e1f35b18\") " pod="openshift-dns/node-resolver-zpmgq"
Nov 04 01:35:16.280111 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280077    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-dbus\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:35:16.280111 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280108    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-etc-kubernetes\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.280336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280124    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-var-lib-kubelet\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.280336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280139    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-sys-fs\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.280336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280155    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-log-socket\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.280336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280169    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-wtmp\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.280336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280183    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-var-lib-cni-bin\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.280336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280203    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-etc-kubernetes\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.280336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280218    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-var-lib-cni-bin\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.280336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280214    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-dbus\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:35:16.280336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280209    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-var-lib-kubelet\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.280336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280242    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-socket-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.280336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280277    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-cni-netd\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.280336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280218    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-log-socket\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.280336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280304    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-var-lib-kubelet\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.280336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280309    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-node-exporter-wtmp\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.280336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280330    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-host\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.280336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280342    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-cni-netd\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280347    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-socket-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280357    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-root\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280264    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-sys-fs\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280367    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-host\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280353    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-var-lib-kubelet\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280381    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-system-cni-dir\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280413    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-root\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280427    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-hostroot\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280502    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-node-log\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280512    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-system-cni-dir\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280501    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-hostroot\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280521    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-node-log\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280519    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-kubelet-config\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280549    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/134fc8ea-90c5-4b4c-86a3-7b83d950f3e5-kubelet-config\") pod \"global-pull-secret-syncer-hjqq9\" (UID: \"134fc8ea-90c5-4b4c-86a3-7b83d950f3e5\") " pod="kube-system/global-pull-secret-syncer-hjqq9"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280554    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aa398d49-e1ee-41df-8785-b765e1f35b18-hosts-file\") pod \"node-resolver-zpmgq\" (UID: \"aa398d49-e1ee-41df-8785-b765e1f35b18\") " pod="openshift-dns/node-resolver-zpmgq"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280577    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-slash\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280578    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aa398d49-e1ee-41df-8785-b765e1f35b18-hosts-file\") pod \"node-resolver-zpmgq\" (UID: \"aa398d49-e1ee-41df-8785-b765e1f35b18\") " pod="openshift-dns/node-resolver-zpmgq"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280598    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-etc-openvswitch\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280616    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-kubelet\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280626    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-slash\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280635    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-device-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280635    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-etc-openvswitch\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280655    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-run-ovn\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280661    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-device-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280669    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-sys\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280673    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-kubelet\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280683    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-sysconfig\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280685    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-run-ovn\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280726    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f-sys\") pod \"node-exporter-27c6h\" (UID: \"4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f\") " pod="openshift-monitoring/node-exporter-27c6h"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280733    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-systemd-units\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280756    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-sysconfig\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280771    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-systemd-units\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280772    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-run\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280793    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-systemd\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280807    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-socket-dir-parent\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280822    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20296e8e-e25f-40e1-9b2b-4971b7e69141-host\") pod \"node-ca-z992q\" (UID: \"20296e8e-e25f-40e1-9b2b-4971b7e69141\") " pod="openshift-image-registry/node-ca-z992q"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280837    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-cni-bin\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280852    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-systemd\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280854    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-socket-dir-parent\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280853    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-sysctl-conf\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280805    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-run\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280871    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20296e8e-e25f-40e1-9b2b-4971b7e69141-host\") pod \"node-ca-z992q\" (UID: \"20296e8e-e25f-40e1-9b2b-4971b7e69141\") " pod="openshift-image-registry/node-ca-z992q"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280880    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-cni-bin\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280887    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-crio-socket\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280911    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-conf-dir\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280934    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8f004fb-8c9c-4efb-b60d-358e32ba5120-system-cni-dir\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280939    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-sysctl-conf\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280956    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce-crio-socket\") pod \"insights-runtime-extractor-8zjpf\" (UID: \"cf2ea8fb-fd39-40ee-8ad5-c2bd3db434ce\") " pod="openshift-insights/insights-runtime-extractor-8zjpf"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280962    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-run-systemd\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280974    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-conf-dir\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280979    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-run-netns\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280994    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-run-ovn-kubernetes\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281003    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-run-systemd\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.280973    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8f004fb-8c9c-4efb-b60d-358e32ba5120-system-cni-dir\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281032    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-run-netns\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281008    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-run-netns\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281045    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-run-ovn-kubernetes\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281059    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-registration-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281041    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-run-netns\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281079    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-cni-dir\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281096    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281108    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-registration-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281123    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-kubernetes\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281143    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-multus-cni-dir\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281145    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281159    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-kubernetes\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.281115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281149    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-sysctl-d\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281194    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-cnibin\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281220    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-modprobe-d\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281227    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-sysctl-d\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281236    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-cnibin\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281260    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52cfdb11-898a-4673-97c5-ecddc1385a03-host-slash\") pod \"iptables-alerter-2sbtc\" (UID: \"52cfdb11-898a-4673-97c5-ecddc1385a03\") " pod="openshift-network-operator/iptables-alerter-2sbtc"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281282    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-etc-modprobe-d\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281284    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8f004fb-8c9c-4efb-b60d-358e32ba5120-cnibin\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281309    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52cfdb11-898a-4673-97c5-ecddc1385a03-host-slash\") pod \"iptables-alerter-2sbtc\" (UID: \"52cfdb11-898a-4673-97c5-ecddc1385a03\") " pod="openshift-network-operator/iptables-alerter-2sbtc"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281312    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-var-lib-cni-multus\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281324    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8f004fb-8c9c-4efb-b60d-358e32ba5120-cnibin\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281335    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-run-multus-certs\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281358    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-var-lib-cni-multus\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281371    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-etc-selinux\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281393    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-run-multus-certs\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281397    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-run-k8s-cni-cncf-io\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281426    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-etc-selinux\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281428    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281470    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-var-lib-openvswitch\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281475    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/67ed6df2-450d-4705-8a11-f98d5b507dc4-host-run-k8s-cni-cncf-io\") pod \"multus-hcvld\" (UID: \"67ed6df2-450d-4705-8a11-f98d5b507dc4\") " pod="openshift-multus/multus-hcvld"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281495    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-var-lib-openvswitch\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281495    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-sys\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281501    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7acf7f6e-efb8-498e-8066-a60fbedecd49-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rp8rx\" (UID: \"7acf7f6e-efb8-498e-8066-a60fbedecd49\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rp8rx"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281518    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-run-openvswitch\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281534    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-lib-modules\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281543    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-sys\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281554    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8f004fb-8c9c-4efb-b60d-358e32ba5120-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281581    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8f004fb-8c9c-4efb-b60d-358e32ba5120-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hh4tb\" (UID: \"a8f004fb-8c9c-4efb-b60d-358e32ba5120\") " pod="openshift-multus/multus-additional-cni-plugins-hh4tb"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281595    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/192e2fec-fefb-4bc5-9516-ed7464963a10-run-openvswitch\") pod \"ovnkube-node-6lfbv\" (UID: \"192e2fec-fefb-4bc5-9516-ed7464963a10\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:16.283054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.281682    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37d34756-13c3-4993-a3d1-07f68eada6b0-lib-modules\") pod \"tuned-hhttg\" (UID: \"37d34756-13c3-4993-a3d1-07f68eada6b0\") " pod="openshift-cluster-node-tuning-operator/tuned-hhttg"
Nov 04 01:35:16.373366 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:16.373346    6116 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:35:17.345365 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:17.345324    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-76b68"
Nov 04 01:35:18.114198 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:18.114164    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd725f9-4be0-4be3-bb6a-898213f63761","Type":"ContainerStarted","Data":"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e"}
Nov 04 01:35:18.114351 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:18.114234    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd725f9-4be0-4be3-bb6a-898213f63761","Type":"ContainerStarted","Data":"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b"}
Nov 04 01:35:18.177348 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:18.177321    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:18.762420 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:18.762379    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:19.056786 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:19.056708    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:19.919729 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:19.919690    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"
Nov 04 01:35:19.924186 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:19.924160    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4mgl"
Nov 04 01:35:20.125438 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:20.125396    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd725f9-4be0-4be3-bb6a-898213f63761","Type":"ContainerStarted","Data":"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404"}
Nov 04 01:35:20.125633 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:20.125488    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd725f9-4be0-4be3-bb6a-898213f63761","Type":"ContainerStarted","Data":"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1"}
Nov 04 01:35:20.125633 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:20.125517    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd725f9-4be0-4be3-bb6a-898213f63761","Type":"ContainerStarted","Data":"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8"}
Nov 04 01:35:20.125633 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:20.125540    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd725f9-4be0-4be3-bb6a-898213f63761","Type":"ContainerStarted","Data":"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd"}
Nov 04 01:35:21.581421 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:21.581364    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:23.183897 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:23.183862    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:35:24.262871 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:24.262840    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:24.263242 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:24.263083    6116 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 01:35:28.767324 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:28.767290    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:28.771288 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:28.771259    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:35:33.569191 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:33.569139    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:33.588480 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:33.588436    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:34.188532 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:34.188501    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:37.745170 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:37.745134    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lfbv"
Nov 04 01:35:39.968538 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:39.968506    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kwbdd"
Nov 04 01:35:41.990307 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:41.990267    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:35:43.186985 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:43.186951    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hbs2c"
Nov 04 01:35:51.137320 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.137280    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 01:35:51.138343 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.138044    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="prometheus" containerID="cri-o://620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b" gracePeriod=600
Nov 04 01:35:51.138636 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.138081    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="thanos-sidecar" containerID="cri-o://7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd" gracePeriod=600
Nov 04 01:35:51.138748 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.138082    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="kube-rbac-proxy-thanos" containerID="cri-o://21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404" gracePeriod=600
Nov 04 01:35:51.138806 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.138123    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="kube-rbac-proxy" containerID="cri-o://009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1" gracePeriod=600
Nov 04 01:35:51.138862 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.138134    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="kube-rbac-proxy-web" containerID="cri-o://9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8" gracePeriod=600
Nov 04 01:35:51.138862 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.138165    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="config-reloader" containerID="cri-o://c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e" gracePeriod=600
Nov 04 01:35:51.396188 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.396127    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:51.457203 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.457163    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-tls\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.457203 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.457207    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-grpc-tls\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.457481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.457228    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-1\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.457481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.457253    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-config\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-config\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.457481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.457280    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-metrics-client-certs\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.457481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.457298    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-kube-rbac-proxy\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.457481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.457319    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-config-out\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.457481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.457349    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-2\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.457481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.457384    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-metrics-client-ca\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.457481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.457422    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-thanos-prometheus-http-client-file\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.457481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.457470    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.457910 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.457515    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-serving-certs-ca-bundle\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.457910 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.457540    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-0\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.457910 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.457570    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-db\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.457910 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.457599    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-kube-api-access-v72sb\" (UniqueName: \"kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-kube-api-access-v72sb\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.457910 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.457632    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.457910 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.457667    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-kubelet-serving-ca-bundle\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.458215 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.458028    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-serving-certs-ca-bundle" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:35:51.458215 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.458096    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-tls-assets\" (UniqueName: \"kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-tls-assets\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.458312 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.458243    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-trusted-ca-bundle\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.458312 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.458276    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-web-config\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-web-config\") pod \"cfd725f9-4be0-4be3-bb6a-898213f63761\" (UID: \"cfd725f9-4be0-4be3-bb6a-898213f63761\") "
Nov 04 01:35:51.458491 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.458441    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-1" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:35:51.458574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.458548    6116 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.458574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.458568    6116 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.458959 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.458892    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-metrics-client-ca" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:35:51.459176 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.459094    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-0" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:35:51.459771 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.459740    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-2" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:35:51.460364 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.460339    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-kubelet-serving-ca-bundle" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:35:51.460705 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.460560    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-db" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:35:51.460705 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.460603    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-metrics-client-certs" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:35:51.461119 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.461076    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-kube-rbac-proxy" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:35:51.461223 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.461158    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-trusted-ca-bundle" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:35:51.461286 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.461232    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-config-out" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:35:51.461726 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.461694    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-config" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:35:51.461861 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.461832    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-grpc-tls" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:35:51.462204 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.462180    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-tls" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:35:51.462695 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.462668    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-thanos-prometheus-http-client-file" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:35:51.462766 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.462741    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-kube-api-access-v72sb" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "kube-api-access-v72sb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:35:51.463173 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.463155    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-thanos-sidecar-tls" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:35:51.463416 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.463399    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-tls-assets" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:35:51.463617 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.463597    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-kube-rbac-proxy-web" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:35:51.471637 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.471612    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-web-config" pod "cfd725f9-4be0-4be3-bb6a-898213f63761" (UID: "cfd725f9-4be0-4be3-bb6a-898213f63761"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:35:51.559488 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.559434    6116 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.559488 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.559491    6116 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-db\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.559697 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.559509    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v72sb\" (UniqueName: \"kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-kube-api-access-v72sb\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.559697 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.559553    6116 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.559697 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.559569    6116 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.559697 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.559584    6116 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cfd725f9-4be0-4be3-bb6a-898213f63761-tls-assets\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.559697 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.559598    6116 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.559697 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.559615    6116 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-web-config\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.559697 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.559628    6116 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.559697 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.559642    6116 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-grpc-tls\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.559697 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.559655    6116 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-config\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.559697 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.559671    6116 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-metrics-client-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.559697 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.559687    6116 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-kube-rbac-proxy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.559697 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.559700    6116 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cfd725f9-4be0-4be3-bb6a-898213f63761-config-out\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.560093 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.559713    6116 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.560093 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.559727    6116 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfd725f9-4be0-4be3-bb6a-898213f63761-configmap-metrics-client-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.560093 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.559744    6116 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:51.560093 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:51.559758    6116 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cfd725f9-4be0-4be3-bb6a-898213f63761-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:35:52.240268 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.240227    6116 generic.go:356] "Generic (PLEG): container finished" podID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerID="21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404" exitCode=0
Nov 04 01:35:52.240268 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.240260    6116 generic.go:356] "Generic (PLEG): container finished" podID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerID="009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1" exitCode=0
Nov 04 01:35:52.240268 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.240269    6116 generic.go:356] "Generic (PLEG): container finished" podID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerID="9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8" exitCode=0
Nov 04 01:35:52.240750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.240278    6116 generic.go:356] "Generic (PLEG): container finished" podID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerID="7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd" exitCode=0
Nov 04 01:35:52.240750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.240286    6116 generic.go:356] "Generic (PLEG): container finished" podID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerID="c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e" exitCode=0
Nov 04 01:35:52.240750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.240294    6116 generic.go:356] "Generic (PLEG): container finished" podID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerID="620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b" exitCode=0
Nov 04 01:35:52.240750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.240302    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd725f9-4be0-4be3-bb6a-898213f63761","Type":"ContainerDied","Data":"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404"}
Nov 04 01:35:52.240750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.240319    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.240750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.240373    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd725f9-4be0-4be3-bb6a-898213f63761","Type":"ContainerDied","Data":"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1"}
Nov 04 01:35:52.240750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.240402    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd725f9-4be0-4be3-bb6a-898213f63761","Type":"ContainerDied","Data":"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8"}
Nov 04 01:35:52.240750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.240425    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd725f9-4be0-4be3-bb6a-898213f63761","Type":"ContainerDied","Data":"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd"}
Nov 04 01:35:52.240750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.240465    6116 scope.go:122] "RemoveContainer" containerID="21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404"
Nov 04 01:35:52.240750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.240467    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd725f9-4be0-4be3-bb6a-898213f63761","Type":"ContainerDied","Data":"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e"}
Nov 04 01:35:52.240750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.240566    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd725f9-4be0-4be3-bb6a-898213f63761","Type":"ContainerDied","Data":"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b"}
Nov 04 01:35:52.240750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.240599    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cfd725f9-4be0-4be3-bb6a-898213f63761","Type":"ContainerDied","Data":"355a1c2d04ae568f6664e694146974b8ab351c306ea8bb7e39207174d85a98bf"}
Nov 04 01:35:52.249721 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.249699    6116 scope.go:122] "RemoveContainer" containerID="009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1"
Nov 04 01:35:52.256487 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.256429    6116 scope.go:122] "RemoveContainer" containerID="9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8"
Nov 04 01:35:52.263252 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.263228    6116 scope.go:122] "RemoveContainer" containerID="7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd"
Nov 04 01:35:52.269916 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.269897    6116 scope.go:122] "RemoveContainer" containerID="c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e"
Nov 04 01:35:52.276660 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.276638    6116 scope.go:122] "RemoveContainer" containerID="620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b"
Nov 04 01:35:52.283841 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.283823    6116 scope.go:122] "RemoveContainer" containerID="a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7"
Nov 04 01:35:52.290650 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.290631    6116 scope.go:122] "RemoveContainer" containerID="21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404"
Nov 04 01:35:52.290931 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:52.290908    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404\": container with ID starting with 21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404 not found: ID does not exist" containerID="21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404"
Nov 04 01:35:52.291008 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.290946    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404"} err="failed to get container status \"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404\": rpc error: code = NotFound desc = could not find container \"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404\": container with ID starting with 21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404 not found: ID does not exist"
Nov 04 01:35:52.291008 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.290992    6116 scope.go:122] "RemoveContainer" containerID="009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1"
Nov 04 01:35:52.291249 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:52.291231    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1\": container with ID starting with 009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1 not found: ID does not exist" containerID="009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1"
Nov 04 01:35:52.291287 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.291257    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1"} err="failed to get container status \"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1\": rpc error: code = NotFound desc = could not find container \"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1\": container with ID starting with 009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1 not found: ID does not exist"
Nov 04 01:35:52.291287 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.291272    6116 scope.go:122] "RemoveContainer" containerID="9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8"
Nov 04 01:35:52.291490 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:52.291473    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8\": container with ID starting with 9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8 not found: ID does not exist" containerID="9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8"
Nov 04 01:35:52.291526 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.291494    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8"} err="failed to get container status \"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8\": rpc error: code = NotFound desc = could not find container \"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8\": container with ID starting with 9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8 not found: ID does not exist"
Nov 04 01:35:52.291526 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.291508    6116 scope.go:122] "RemoveContainer" containerID="7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd"
Nov 04 01:35:52.291719 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:52.291704    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd\": container with ID starting with 7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd not found: ID does not exist" containerID="7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd"
Nov 04 01:35:52.291772 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.291727    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd"} err="failed to get container status \"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd\": rpc error: code = NotFound desc = could not find container \"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd\": container with ID starting with 7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd not found: ID does not exist"
Nov 04 01:35:52.291772 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.291747    6116 scope.go:122] "RemoveContainer" containerID="c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e"
Nov 04 01:35:52.291963 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:52.291949    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e\": container with ID starting with c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e not found: ID does not exist" containerID="c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e"
Nov 04 01:35:52.292000 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.291968    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e"} err="failed to get container status \"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e\": rpc error: code = NotFound desc = could not find container \"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e\": container with ID starting with c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e not found: ID does not exist"
Nov 04 01:35:52.292000 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.291981    6116 scope.go:122] "RemoveContainer" containerID="620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b"
Nov 04 01:35:52.292165 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:52.292151    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b\": container with ID starting with 620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b not found: ID does not exist" containerID="620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b"
Nov 04 01:35:52.292202 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.292168    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b"} err="failed to get container status \"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b\": rpc error: code = NotFound desc = could not find container \"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b\": container with ID starting with 620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b not found: ID does not exist"
Nov 04 01:35:52.292202 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.292179    6116 scope.go:122] "RemoveContainer" containerID="a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7"
Nov 04 01:35:52.292349 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:35:52.292335    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7\": container with ID starting with a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7 not found: ID does not exist" containerID="a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7"
Nov 04 01:35:52.292386 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.292351    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7"} err="failed to get container status \"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7\": rpc error: code = NotFound desc = could not find container \"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7\": container with ID starting with a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7 not found: ID does not exist"
Nov 04 01:35:52.292386 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.292361    6116 scope.go:122] "RemoveContainer" containerID="21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404"
Nov 04 01:35:52.292601 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.292583    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404"} err="failed to get container status \"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404\": rpc error: code = NotFound desc = could not find container \"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404\": container with ID starting with 21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404 not found: ID does not exist"
Nov 04 01:35:52.292660 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.292600    6116 scope.go:122] "RemoveContainer" containerID="009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1"
Nov 04 01:35:52.292827 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.292812    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1"} err="failed to get container status \"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1\": rpc error: code = NotFound desc = could not find container \"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1\": container with ID starting with 009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1 not found: ID does not exist"
Nov 04 01:35:52.292878 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.292827    6116 scope.go:122] "RemoveContainer" containerID="9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8"
Nov 04 01:35:52.293041 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.293017    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8"} err="failed to get container status \"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8\": rpc error: code = NotFound desc = could not find container \"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8\": container with ID starting with 9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8 not found: ID does not exist"
Nov 04 01:35:52.293114 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.293042    6116 scope.go:122] "RemoveContainer" containerID="7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd"
Nov 04 01:35:52.293248 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.293232    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd"} err="failed to get container status \"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd\": rpc error: code = NotFound desc = could not find container \"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd\": container with ID starting with 7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd not found: ID does not exist"
Nov 04 01:35:52.293312 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.293249    6116 scope.go:122] "RemoveContainer" containerID="c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e"
Nov 04 01:35:52.293520 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.293442    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e"} err="failed to get container status \"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e\": rpc error: code = NotFound desc = could not find container \"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e\": container with ID starting with c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e not found: ID does not exist"
Nov 04 01:35:52.293520 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.293481    6116 scope.go:122] "RemoveContainer" containerID="620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b"
Nov 04 01:35:52.293684 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.293667    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b"} err="failed to get container status \"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b\": rpc error: code = NotFound desc = could not find container \"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b\": container with ID starting with 620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b not found: ID does not exist"
Nov 04 01:35:52.293751 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.293684    6116 scope.go:122] "RemoveContainer" containerID="a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7"
Nov 04 01:35:52.293888 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.293869    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7"} err="failed to get container status \"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7\": rpc error: code = NotFound desc = could not find container \"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7\": container with ID starting with a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7 not found: ID does not exist"
Nov 04 01:35:52.293931 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.293889    6116 scope.go:122] "RemoveContainer" containerID="21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404"
Nov 04 01:35:52.294070 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.294054    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404"} err="failed to get container status \"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404\": rpc error: code = NotFound desc = could not find container \"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404\": container with ID starting with 21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404 not found: ID does not exist"
Nov 04 01:35:52.294133 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.294070    6116 scope.go:122] "RemoveContainer" containerID="009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1"
Nov 04 01:35:52.294266 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.294251    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1"} err="failed to get container status \"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1\": rpc error: code = NotFound desc = could not find container \"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1\": container with ID starting with 009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1 not found: ID does not exist"
Nov 04 01:35:52.294312 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.294266    6116 scope.go:122] "RemoveContainer" containerID="9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8"
Nov 04 01:35:52.294491 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.294474    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8"} err="failed to get container status \"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8\": rpc error: code = NotFound desc = could not find container \"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8\": container with ID starting with 9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8 not found: ID does not exist"
Nov 04 01:35:52.294565 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.294491    6116 scope.go:122] "RemoveContainer" containerID="7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd"
Nov 04 01:35:52.294701 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.294684    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd"} err="failed to get container status \"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd\": rpc error: code = NotFound desc = could not find container \"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd\": container with ID starting with 7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd not found: ID does not exist"
Nov 04 01:35:52.294745 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.294702    6116 scope.go:122] "RemoveContainer" containerID="c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e"
Nov 04 01:35:52.294881 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.294865    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e"} err="failed to get container status \"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e\": rpc error: code = NotFound desc = could not find container \"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e\": container with ID starting with c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e not found: ID does not exist"
Nov 04 01:35:52.294961 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.294882    6116 scope.go:122] "RemoveContainer" containerID="620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b"
Nov 04 01:35:52.295085 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.295069    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b"} err="failed to get container status \"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b\": rpc error: code = NotFound desc = could not find container \"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b\": container with ID starting with 620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b not found: ID does not exist"
Nov 04 01:35:52.295129 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.295086    6116 scope.go:122] "RemoveContainer" containerID="a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7"
Nov 04 01:35:52.295264 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.295250    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7"} err="failed to get container status \"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7\": rpc error: code = NotFound desc = could not find container \"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7\": container with ID starting with a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7 not found: ID does not exist"
Nov 04 01:35:52.295307 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.295265    6116 scope.go:122] "RemoveContainer" containerID="21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404"
Nov 04 01:35:52.295588 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.295567    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404"} err="failed to get container status \"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404\": rpc error: code = NotFound desc = could not find container \"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404\": container with ID starting with 21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404 not found: ID does not exist"
Nov 04 01:35:52.295638 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.295590    6116 scope.go:122] "RemoveContainer" containerID="009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1"
Nov 04 01:35:52.295824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.295808    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1"} err="failed to get container status \"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1\": rpc error: code = NotFound desc = could not find container \"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1\": container with ID starting with 009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1 not found: ID does not exist"
Nov 04 01:35:52.295875 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.295824    6116 scope.go:122] "RemoveContainer" containerID="9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8"
Nov 04 01:35:52.296014 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.296000    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8"} err="failed to get container status \"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8\": rpc error: code = NotFound desc = could not find container \"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8\": container with ID starting with 9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8 not found: ID does not exist"
Nov 04 01:35:52.296014 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.296012    6116 scope.go:122] "RemoveContainer" containerID="7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd"
Nov 04 01:35:52.296222 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.296204    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd"} err="failed to get container status \"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd\": rpc error: code = NotFound desc = could not find container \"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd\": container with ID starting with 7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd not found: ID does not exist"
Nov 04 01:35:52.296285 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.296223    6116 scope.go:122] "RemoveContainer" containerID="c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e"
Nov 04 01:35:52.296471 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.296436    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e"} err="failed to get container status \"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e\": rpc error: code = NotFound desc = could not find container \"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e\": container with ID starting with c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e not found: ID does not exist"
Nov 04 01:35:52.296509 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.296471    6116 scope.go:122] "RemoveContainer" containerID="620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b"
Nov 04 01:35:52.296683 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.296667    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b"} err="failed to get container status \"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b\": rpc error: code = NotFound desc = could not find container \"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b\": container with ID starting with 620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b not found: ID does not exist"
Nov 04 01:35:52.296750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.296684    6116 scope.go:122] "RemoveContainer" containerID="a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7"
Nov 04 01:35:52.296912 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.296890    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7"} err="failed to get container status \"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7\": rpc error: code = NotFound desc = could not find container \"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7\": container with ID starting with a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7 not found: ID does not exist"
Nov 04 01:35:52.296912 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.296909    6116 scope.go:122] "RemoveContainer" containerID="21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404"
Nov 04 01:35:52.297120 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.297104    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404"} err="failed to get container status \"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404\": rpc error: code = NotFound desc = could not find container \"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404\": container with ID starting with 21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404 not found: ID does not exist"
Nov 04 01:35:52.297120 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.297121    6116 scope.go:122] "RemoveContainer" containerID="009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1"
Nov 04 01:35:52.297340 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.297322    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1"} err="failed to get container status \"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1\": rpc error: code = NotFound desc = could not find container \"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1\": container with ID starting with 009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1 not found: ID does not exist"
Nov 04 01:35:52.297390 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.297340    6116 scope.go:122] "RemoveContainer" containerID="9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8"
Nov 04 01:35:52.297564 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.297544    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8"} err="failed to get container status \"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8\": rpc error: code = NotFound desc = could not find container \"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8\": container with ID starting with 9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8 not found: ID does not exist"
Nov 04 01:35:52.297613 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.297564    6116 scope.go:122] "RemoveContainer" containerID="7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd"
Nov 04 01:35:52.297824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.297801    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd"} err="failed to get container status \"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd\": rpc error: code = NotFound desc = could not find container \"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd\": container with ID starting with 7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd not found: ID does not exist"
Nov 04 01:35:52.297824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.297819    6116 scope.go:122] "RemoveContainer" containerID="c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e"
Nov 04 01:35:52.298140 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.298087    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e"} err="failed to get container status \"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e\": rpc error: code = NotFound desc = could not find container \"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e\": container with ID starting with c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e not found: ID does not exist"
Nov 04 01:35:52.298193 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.298144    6116 scope.go:122] "RemoveContainer" containerID="620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b"
Nov 04 01:35:52.298393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.298374    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b"} err="failed to get container status \"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b\": rpc error: code = NotFound desc = could not find container \"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b\": container with ID starting with 620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b not found: ID does not exist"
Nov 04 01:35:52.298490 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.298394    6116 scope.go:122] "RemoveContainer" containerID="a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7"
Nov 04 01:35:52.298675 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.298656    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7"} err="failed to get container status \"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7\": rpc error: code = NotFound desc = could not find container \"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7\": container with ID starting with a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7 not found: ID does not exist"
Nov 04 01:35:52.298723 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.298679    6116 scope.go:122] "RemoveContainer" containerID="21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404"
Nov 04 01:35:52.298888 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.298873    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404"} err="failed to get container status \"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404\": rpc error: code = NotFound desc = could not find container \"21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404\": container with ID starting with 21daf4556f29528e14d1920765e93bfd48e62df55e3b70b80be8846362f7d404 not found: ID does not exist"
Nov 04 01:35:52.298925 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.298896    6116 scope.go:122] "RemoveContainer" containerID="009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1"
Nov 04 01:35:52.299108 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.299091    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1"} err="failed to get container status \"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1\": rpc error: code = NotFound desc = could not find container \"009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1\": container with ID starting with 009dcaaf26e79304abc2895d5635725a96bba19430d673fa9dbe9ca8e99c10e1 not found: ID does not exist"
Nov 04 01:35:52.299187 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.299109    6116 scope.go:122] "RemoveContainer" containerID="9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8"
Nov 04 01:35:52.299341 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.299320    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8"} err="failed to get container status \"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8\": rpc error: code = NotFound desc = could not find container \"9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8\": container with ID starting with 9520c7ab5d63b8923c8e363e504f015b6a42b84e680b19447a3201fb4a8f93f8 not found: ID does not exist"
Nov 04 01:35:52.299388 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.299342    6116 scope.go:122] "RemoveContainer" containerID="7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd"
Nov 04 01:35:52.299596 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.299581    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd"} err="failed to get container status \"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd\": rpc error: code = NotFound desc = could not find container \"7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd\": container with ID starting with 7f8cefcc845f903fec682e0c5358857669e57942e7a86cb0b27ae9aba194a4dd not found: ID does not exist"
Nov 04 01:35:52.299596 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.299595    6116 scope.go:122] "RemoveContainer" containerID="c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e"
Nov 04 01:35:52.299801 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.299782    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e"} err="failed to get container status \"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e\": rpc error: code = NotFound desc = could not find container \"c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e\": container with ID starting with c372c99ef0fcf90298b0e913c2c92de34014d4451a6d4c82892f965ab1845f2e not found: ID does not exist"
Nov 04 01:35:52.299841 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.299801    6116 scope.go:122] "RemoveContainer" containerID="620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b"
Nov 04 01:35:52.299993 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.299977    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b"} err="failed to get container status \"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b\": rpc error: code = NotFound desc = could not find container \"620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b\": container with ID starting with 620027a8f19f24e73af80c997082e226bba907ccd568422b24c0a5d0a7b39a4b not found: ID does not exist"
Nov 04 01:35:52.300033 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.299994    6116 scope.go:122] "RemoveContainer" containerID="a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7"
Nov 04 01:35:52.300183 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.300165    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7"} err="failed to get container status \"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7\": rpc error: code = NotFound desc = could not find container \"a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7\": container with ID starting with a9273c0eb2ccfc234ae73d12c71c34c235248d65c3e5e9a96cf320503306b5b7 not found: ID does not exist"
Nov 04 01:35:52.456401 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.456325    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 01:35:52.506046 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506016    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 01:35:52.506291 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506274    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 01:35:52.506613 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506590    6116 cpu_manager.go:414] "containerMap: removing container" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="config-reloader"
Nov 04 01:35:52.506613 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506612    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="config-reloader"
Nov 04 01:35:52.506795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506628    6116 cpu_manager.go:414] "containerMap: removing container" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="kube-rbac-proxy"
Nov 04 01:35:52.506795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506635    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="kube-rbac-proxy"
Nov 04 01:35:52.506795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506646    6116 cpu_manager.go:414] "containerMap: removing container" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="kube-rbac-proxy-thanos"
Nov 04 01:35:52.506795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506653    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="kube-rbac-proxy-thanos"
Nov 04 01:35:52.506795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506660    6116 cpu_manager.go:414] "containerMap: removing container" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="init-config-reloader"
Nov 04 01:35:52.506795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506665    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="init-config-reloader"
Nov 04 01:35:52.506795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506704    6116 cpu_manager.go:414] "containerMap: removing container" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="prometheus"
Nov 04 01:35:52.506795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506709    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="prometheus"
Nov 04 01:35:52.506795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506721    6116 cpu_manager.go:414] "containerMap: removing container" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="kube-rbac-proxy-web"
Nov 04 01:35:52.506795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506726    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="kube-rbac-proxy-web"
Nov 04 01:35:52.506795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506735    6116 cpu_manager.go:414] "containerMap: removing container" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="thanos-sidecar"
Nov 04 01:35:52.506795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506739    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="thanos-sidecar"
Nov 04 01:35:52.506795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506786    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="config-reloader"
Nov 04 01:35:52.506795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506793    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="kube-rbac-proxy"
Nov 04 01:35:52.506795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506799    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="kube-rbac-proxy-thanos"
Nov 04 01:35:52.507350 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506808    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="prometheus"
Nov 04 01:35:52.507350 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506817    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="kube-rbac-proxy-web"
Nov 04 01:35:52.507350 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.506825    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" containerName="thanos-sidecar"
Nov 04 01:35:52.509891 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.509875    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.562818 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.562786    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Nov 04 01:35:52.562818 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.562804    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Nov 04 01:35:52.563052 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.562937    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Nov 04 01:35:52.563256 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.563167    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Nov 04 01:35:52.563256 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.563178    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Nov 04 01:35:52.563256 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.563228    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Nov 04 01:35:52.563256 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.563237    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 04 01:35:52.563605 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.563559    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Nov 04 01:35:52.563605 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.563569    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Nov 04 01:35:52.563605 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.563565    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Nov 04 01:35:52.563822 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.563699    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Nov 04 01:35:52.563822 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.563716    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2bmnb\""
Nov 04 01:35:52.563822 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.563728    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Nov 04 01:35:52.563822 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.563744    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Nov 04 01:35:52.565893 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.565868    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d586274f-5bbb-468e-9eba-3d6809e4c78a-config-out\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566004 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.565897    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566004 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.565915    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566004 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.565932    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566004 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.565996    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566202 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.566029    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566202 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.566079    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566202 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.566190    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566334 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.566219    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566334 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.566253    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566334 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.566276    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxmwx\" (UniqueName: \"kubernetes.io/projected/d586274f-5bbb-468e-9eba-3d6809e4c78a-kube-api-access-bxmwx\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566438 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.566346    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566438 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.566365    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d586274f-5bbb-468e-9eba-3d6809e4c78a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566438 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.566400    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566603 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.566438    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566603 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.566506    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566603 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.566545    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566715 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.566606    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-config\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566715 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.566635    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d586274f-5bbb-468e-9eba-3d6809e4c78a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.566715 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.566654    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-web-config\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.612971 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.612935    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1eqn9lagl0d1b\""
Nov 04 01:35:52.613557 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.613535    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Nov 04 01:35:52.613641 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.613578    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Nov 04 01:35:52.667820 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.667788    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-config\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.667820 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.667826    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d586274f-5bbb-468e-9eba-3d6809e4c78a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.668072 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.667842    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-web-config\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.668072 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.667879    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d586274f-5bbb-468e-9eba-3d6809e4c78a-config-out\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.668072 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.667908    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.668072 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.667931    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.668072 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.667954    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.668072 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.667984    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.668072 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.668004    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.668072 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.668026    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.668072 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.668060    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.668505 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.668081    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.668505 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.668110    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.668505 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.668131    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxmwx\" (UniqueName: \"kubernetes.io/projected/d586274f-5bbb-468e-9eba-3d6809e4c78a-kube-api-access-bxmwx\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.668505 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.668169    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.668505 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.668193    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d586274f-5bbb-468e-9eba-3d6809e4c78a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.668505 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.668217    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.668505 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.668233    6116 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 01:35:52.668505 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.668247    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.668505 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.668271    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.668505 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.668294    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.669062 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.668780    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.669062 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.669027    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.669549 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.669409    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.669549 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.669437    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.669877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.669759    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d586274f-5bbb-468e-9eba-3d6809e4c78a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.671307 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.671270    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d586274f-5bbb-468e-9eba-3d6809e4c78a-config-out\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.671482 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.671327    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.671590 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.671512    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.671590 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.671517    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.671844 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.671811    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.671944 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.671922    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-config\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.672275 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.672253    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-web-config\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.672500 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.672473    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.673949 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.673926    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.674842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.674412    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.674842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.674424    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.674842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.674757    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d586274f-5bbb-468e-9eba-3d6809e4c78a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.675612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.675592    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d586274f-5bbb-468e-9eba-3d6809e4c78a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.675777 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.675759    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d586274f-5bbb-468e-9eba-3d6809e4c78a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.732466 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.732373    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxmwx\" (UniqueName: \"kubernetes.io/projected/d586274f-5bbb-468e-9eba-3d6809e4c78a-kube-api-access-bxmwx\") pod \"prometheus-k8s-0\" (UID: \"d586274f-5bbb-468e-9eba-3d6809e4c78a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:52.784229 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.784178    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 01:35:52.825785 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:52.825740    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:53.024977 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:53.024891    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cfd725f9-4be0-4be3-bb6a-898213f63761" path="/var/lib/kubelet/pods/cfd725f9-4be0-4be3-bb6a-898213f63761/volumes"
Nov 04 01:35:53.255775 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:53.255743    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 01:35:53.259167 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:35:53.259140    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd586274f_5bbb_468e_9eba_3d6809e4c78a.slice/crio-1f666e1d280585fa75cc2b381eb0c6a27d5df89509e415446022341a7f13f495 WatchSource:0}: Error finding container 1f666e1d280585fa75cc2b381eb0c6a27d5df89509e415446022341a7f13f495: Status 404 returned error can't find the container with id 1f666e1d280585fa75cc2b381eb0c6a27d5df89509e415446022341a7f13f495
Nov 04 01:35:54.248774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:54.248737    6116 generic.go:356] "Generic (PLEG): container finished" podID="d586274f-5bbb-468e-9eba-3d6809e4c78a" containerID="061971baff668e0f7cd9b64ea58669e5a7ed27000dabdf408b959d6142c941f7" exitCode=0
Nov 04 01:35:54.248978 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:54.248810    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d586274f-5bbb-468e-9eba-3d6809e4c78a","Type":"ContainerDied","Data":"061971baff668e0f7cd9b64ea58669e5a7ed27000dabdf408b959d6142c941f7"}
Nov 04 01:35:54.248978 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:54.248887    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d586274f-5bbb-468e-9eba-3d6809e4c78a","Type":"ContainerStarted","Data":"1f666e1d280585fa75cc2b381eb0c6a27d5df89509e415446022341a7f13f495"}
Nov 04 01:35:55.256610 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:55.256579    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/prometheus/0.log"
Nov 04 01:35:55.257004 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:55.256958    6116 generic.go:356] "Generic (PLEG): container finished" podID="d586274f-5bbb-468e-9eba-3d6809e4c78a" containerID="f0d95376bae9ad9060c9666182f8e69235b4be9ae79651c14e28e7cb23a75f82" exitCode=2
Nov 04 01:35:55.257061 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:55.257014    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d586274f-5bbb-468e-9eba-3d6809e4c78a","Type":"ContainerStarted","Data":"07b162f0e291399bbe91ab069f680887d038e465b17cb3136e60224e63bcad0b"}
Nov 04 01:35:55.257153 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:55.257137    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d586274f-5bbb-468e-9eba-3d6809e4c78a","Type":"ContainerStarted","Data":"7b6d92a89a5b1570ff76eed855a91329c7c13e67d6f52dcdcfe97321eb02ecc0"}
Nov 04 01:35:55.257196 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:55.257184    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d586274f-5bbb-468e-9eba-3d6809e4c78a","Type":"ContainerStarted","Data":"72eda13fc482b7ef7adc5367cb58173484533ea2d75cf112c418a6ee4116fb24"}
Nov 04 01:35:55.257245 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:55.257234    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d586274f-5bbb-468e-9eba-3d6809e4c78a","Type":"ContainerStarted","Data":"497fb195763cf4ed5ded06582b2ce44f3aa23131b57e7de9e61ee1fc3be7e976"}
Nov 04 01:35:55.257285 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:55.257274    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d586274f-5bbb-468e-9eba-3d6809e4c78a","Type":"ContainerStarted","Data":"b505add46843d20cfdb90c76678ab23fc99667f5d8570a8d97dd62527b68c517"}
Nov 04 01:35:55.257339 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:55.257328    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d586274f-5bbb-468e-9eba-3d6809e4c78a","Type":"ContainerDied","Data":"f0d95376bae9ad9060c9666182f8e69235b4be9ae79651c14e28e7cb23a75f82"}
Nov 04 01:35:55.257516 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:55.257504    6116 scope.go:122] "RemoveContainer" containerID="f0d95376bae9ad9060c9666182f8e69235b4be9ae79651c14e28e7cb23a75f82"
Nov 04 01:35:56.263923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:56.263899    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/prometheus/0.log"
Nov 04 01:35:56.264346 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:56.264287    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d586274f-5bbb-468e-9eba-3d6809e4c78a","Type":"ContainerStarted","Data":"60e4ed3a1444f32435b16048be2cccbae9248b0d26e03f56fc0452fd0049fea3"}
Nov 04 01:35:56.545174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:56.545044    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.530928394 podStartE2EDuration="4.545027908s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.099514ms" isStatefulPod=false podCreationTimestamp="2026-11-04 01:35:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:35:56.495535217 +0000 UTC m=+41.616812735" watchObservedRunningTime="2026-11-04 01:35:56.545027908 +0000 UTC m=+41.666305428"
Nov 04 01:35:57.826177 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:57.826141    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:57.917871 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:57.917839    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:57.933877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:57.933849    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:35:58.287220 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:35:58.287194    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:36:04.062102 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:04.062068    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cdfff847d-84rl8"]
Nov 04 01:36:11.990849 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:36:11.990806    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:36:29.081657 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.081618    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6cdfff847d-84rl8" podUID="010e4cd1-587e-43a7-9f58-93ca75a074df" containerName="console" containerID="cri-o://1f0b7dc8e7c491c3a0a47bf377f8f668e89df1ffd42cfe2a1eeda896853a15d1" gracePeriod=15
Nov 04 01:36:29.329363 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.329336    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:36:29.370817 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.370738    6116 generic.go:356] "Generic (PLEG): container finished" podID="010e4cd1-587e-43a7-9f58-93ca75a074df" containerID="1f0b7dc8e7c491c3a0a47bf377f8f668e89df1ffd42cfe2a1eeda896853a15d1" exitCode=0
Nov 04 01:36:29.370817 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.370794    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cdfff847d-84rl8"
Nov 04 01:36:29.371031 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.370833    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cdfff847d-84rl8" event={"ID":"010e4cd1-587e-43a7-9f58-93ca75a074df","Type":"ContainerDied","Data":"1f0b7dc8e7c491c3a0a47bf377f8f668e89df1ffd42cfe2a1eeda896853a15d1"}
Nov 04 01:36:29.371031 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.370923    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cdfff847d-84rl8" event={"ID":"010e4cd1-587e-43a7-9f58-93ca75a074df","Type":"ContainerDied","Data":"46bc5e27140f7feda0532059508a8f407c36ea862d2005918b83dd88f2b0c7f6"}
Nov 04 01:36:29.371031 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.370951    6116 scope.go:122] "RemoveContainer" containerID="1f0b7dc8e7c491c3a0a47bf377f8f668e89df1ffd42cfe2a1eeda896853a15d1"
Nov 04 01:36:29.378691 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.378673    6116 scope.go:122] "RemoveContainer" containerID="1f0b7dc8e7c491c3a0a47bf377f8f668e89df1ffd42cfe2a1eeda896853a15d1"
Nov 04 01:36:29.378959 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:36:29.378939    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f0b7dc8e7c491c3a0a47bf377f8f668e89df1ffd42cfe2a1eeda896853a15d1\": container with ID starting with 1f0b7dc8e7c491c3a0a47bf377f8f668e89df1ffd42cfe2a1eeda896853a15d1 not found: ID does not exist" containerID="1f0b7dc8e7c491c3a0a47bf377f8f668e89df1ffd42cfe2a1eeda896853a15d1"
Nov 04 01:36:29.379028 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.378968    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f0b7dc8e7c491c3a0a47bf377f8f668e89df1ffd42cfe2a1eeda896853a15d1"} err="failed to get container status \"1f0b7dc8e7c491c3a0a47bf377f8f668e89df1ffd42cfe2a1eeda896853a15d1\": rpc error: code = NotFound desc = could not find container \"1f0b7dc8e7c491c3a0a47bf377f8f668e89df1ffd42cfe2a1eeda896853a15d1\": container with ID starting with 1f0b7dc8e7c491c3a0a47bf377f8f668e89df1ffd42cfe2a1eeda896853a15d1 not found: ID does not exist"
Nov 04 01:36:29.381217 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.381202    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-trusted-ca-bundle\") pod \"010e4cd1-587e-43a7-9f58-93ca75a074df\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") "
Nov 04 01:36:29.381269 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.381254    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/010e4cd1-587e-43a7-9f58-93ca75a074df-kube-api-access-4nrq5\" (UniqueName: \"kubernetes.io/projected/010e4cd1-587e-43a7-9f58-93ca75a074df-kube-api-access-4nrq5\") pod \"010e4cd1-587e-43a7-9f58-93ca75a074df\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") "
Nov 04 01:36:29.381318 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.381296    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-console-config\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-console-config\") pod \"010e4cd1-587e-43a7-9f58-93ca75a074df\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") "
Nov 04 01:36:29.381367 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.381325    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-service-ca\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-service-ca\") pod \"010e4cd1-587e-43a7-9f58-93ca75a074df\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") "
Nov 04 01:36:29.381367 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.381352    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/010e4cd1-587e-43a7-9f58-93ca75a074df-tmp\" (UniqueName: \"kubernetes.io/empty-dir/010e4cd1-587e-43a7-9f58-93ca75a074df-tmp\") pod \"010e4cd1-587e-43a7-9f58-93ca75a074df\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") "
Nov 04 01:36:29.381537 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.381401    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-oauth-serving-cert\") pod \"010e4cd1-587e-43a7-9f58-93ca75a074df\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") "
Nov 04 01:36:29.381537 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.381486    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-serving-cert\") pod \"010e4cd1-587e-43a7-9f58-93ca75a074df\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") "
Nov 04 01:36:29.381537 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.381528    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-oauth-config\") pod \"010e4cd1-587e-43a7-9f58-93ca75a074df\" (UID: \"010e4cd1-587e-43a7-9f58-93ca75a074df\") "
Nov 04 01:36:29.381681 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.381626    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-trusted-ca-bundle" pod "010e4cd1-587e-43a7-9f58-93ca75a074df" (UID: "010e4cd1-587e-43a7-9f58-93ca75a074df"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:36:29.381751 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.381713    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/010e4cd1-587e-43a7-9f58-93ca75a074df-tmp" pod "010e4cd1-587e-43a7-9f58-93ca75a074df" (UID: "010e4cd1-587e-43a7-9f58-93ca75a074df"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:36:29.381869 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.381839    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-service-ca" pod "010e4cd1-587e-43a7-9f58-93ca75a074df" (UID: "010e4cd1-587e-43a7-9f58-93ca75a074df"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:36:29.381869 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.381854    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-oauth-serving-cert" pod "010e4cd1-587e-43a7-9f58-93ca75a074df" (UID: "010e4cd1-587e-43a7-9f58-93ca75a074df"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:36:29.382198 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.381863    6116 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-trusted-ca-bundle\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:36:29.382198 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.381899    6116 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/010e4cd1-587e-43a7-9f58-93ca75a074df-tmp\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:36:29.382198 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.382004    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-console-config" pod "010e4cd1-587e-43a7-9f58-93ca75a074df" (UID: "010e4cd1-587e-43a7-9f58-93ca75a074df"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:36:29.383666 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.383642    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/010e4cd1-587e-43a7-9f58-93ca75a074df-kube-api-access-4nrq5" pod "010e4cd1-587e-43a7-9f58-93ca75a074df" (UID: "010e4cd1-587e-43a7-9f58-93ca75a074df"). InnerVolumeSpecName "kube-api-access-4nrq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:36:29.383811 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.383793    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-oauth-config" pod "010e4cd1-587e-43a7-9f58-93ca75a074df" (UID: "010e4cd1-587e-43a7-9f58-93ca75a074df"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:36:29.383867 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.383843    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-serving-cert" pod "010e4cd1-587e-43a7-9f58-93ca75a074df" (UID: "010e4cd1-587e-43a7-9f58-93ca75a074df"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:36:29.483198 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.483158    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nrq5\" (UniqueName: \"kubernetes.io/projected/010e4cd1-587e-43a7-9f58-93ca75a074df-kube-api-access-4nrq5\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:36:29.483198 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.483198    6116 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-console-config\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:36:29.483411 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.483211    6116 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-service-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:36:29.483411 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.483220    6116 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/010e4cd1-587e-43a7-9f58-93ca75a074df-oauth-serving-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:36:29.483411 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.483230    6116 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-serving-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:36:29.483411 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.483240    6116 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/010e4cd1-587e-43a7-9f58-93ca75a074df-console-oauth-config\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:36:29.790160 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.790125    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cdfff847d-84rl8"]
Nov 04 01:36:29.793673 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:29.793650    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6cdfff847d-84rl8"]
Nov 04 01:36:31.024604 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:36:31.024574    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="010e4cd1-587e-43a7-9f58-93ca75a074df" path="/var/lib/kubelet/pods/010e4cd1-587e-43a7-9f58-93ca75a074df/volumes"
Nov 04 01:36:41.989640 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:36:41.989592    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:37:11.989331 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:37:11.989289    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:37:41.990352 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:37:41.990313    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:37:56.016268 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.016232    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8"]
Nov 04 01:37:56.016783 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.016588    6116 cpu_manager.go:414] "containerMap: removing container" podUID="010e4cd1-587e-43a7-9f58-93ca75a074df" containerName="console"
Nov 04 01:37:56.016783 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.016603    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="010e4cd1-587e-43a7-9f58-93ca75a074df" containerName="console"
Nov 04 01:37:56.016783 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.016664    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="010e4cd1-587e-43a7-9f58-93ca75a074df" containerName="console"
Nov 04 01:37:56.019608 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.019588    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8"
Nov 04 01:37:56.112144 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.112109    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f78843c8-7092-4753-a69f-592ab02bd9b8-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8\" (UID: \"f78843c8-7092-4753-a69f-592ab02bd9b8\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8"
Nov 04 01:37:56.112144 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.112145    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f78843c8-7092-4753-a69f-592ab02bd9b8-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8\" (UID: \"f78843c8-7092-4753-a69f-592ab02bd9b8\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8"
Nov 04 01:37:56.112351 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.112183    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8l7d\" (UniqueName: \"kubernetes.io/projected/f78843c8-7092-4753-a69f-592ab02bd9b8-kube-api-access-s8l7d\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8\" (UID: \"f78843c8-7092-4753-a69f-592ab02bd9b8\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8"
Nov 04 01:37:56.121347 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.121315    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 04 01:37:56.121347 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.121316    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 04 01:37:56.121556 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.121372    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-59lpg\""
Nov 04 01:37:56.122031 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.122012    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8"]
Nov 04 01:37:56.212919 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.212884    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f78843c8-7092-4753-a69f-592ab02bd9b8-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8\" (UID: \"f78843c8-7092-4753-a69f-592ab02bd9b8\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8"
Nov 04 01:37:56.213102 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.212942    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f78843c8-7092-4753-a69f-592ab02bd9b8-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8\" (UID: \"f78843c8-7092-4753-a69f-592ab02bd9b8\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8"
Nov 04 01:37:56.213102 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.212980    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8l7d\" (UniqueName: \"kubernetes.io/projected/f78843c8-7092-4753-a69f-592ab02bd9b8-kube-api-access-s8l7d\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8\" (UID: \"f78843c8-7092-4753-a69f-592ab02bd9b8\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8"
Nov 04 01:37:56.213267 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.213249    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f78843c8-7092-4753-a69f-592ab02bd9b8-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8\" (UID: \"f78843c8-7092-4753-a69f-592ab02bd9b8\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8"
Nov 04 01:37:56.213347 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.213331    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f78843c8-7092-4753-a69f-592ab02bd9b8-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8\" (UID: \"f78843c8-7092-4753-a69f-592ab02bd9b8\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8"
Nov 04 01:37:56.274524 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.274442    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8l7d\" (UniqueName: \"kubernetes.io/projected/f78843c8-7092-4753-a69f-592ab02bd9b8-kube-api-access-s8l7d\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8\" (UID: \"f78843c8-7092-4753-a69f-592ab02bd9b8\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8"
Nov 04 01:37:56.334574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.334545    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8"
Nov 04 01:37:56.705045 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:56.705012    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8"]
Nov 04 01:37:56.708791 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:37:56.708764    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf78843c8_7092_4753_a69f_592ab02bd9b8.slice/crio-7fada0ad9f5c547610f406bee3950c0b42c06226e38453b2563beea2efb84f50 WatchSource:0}: Error finding container 7fada0ad9f5c547610f406bee3950c0b42c06226e38453b2563beea2efb84f50: Status 404 returned error can't find the container with id 7fada0ad9f5c547610f406bee3950c0b42c06226e38453b2563beea2efb84f50
Nov 04 01:37:57.273987 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.273954    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr"]
Nov 04 01:37:57.278594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.278570    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr"
Nov 04 01:37:57.320973 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.320937    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9872ad2f-0534-46a9-8aa2-b23b598543d1-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr\" (UID: \"9872ad2f-0534-46a9-8aa2-b23b598543d1\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr"
Nov 04 01:37:57.321137 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.320981    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9872ad2f-0534-46a9-8aa2-b23b598543d1-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr\" (UID: \"9872ad2f-0534-46a9-8aa2-b23b598543d1\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr"
Nov 04 01:37:57.321137 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.321107    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2s95\" (UniqueName: \"kubernetes.io/projected/9872ad2f-0534-46a9-8aa2-b23b598543d1-kube-api-access-m2s95\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr\" (UID: \"9872ad2f-0534-46a9-8aa2-b23b598543d1\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr"
Nov 04 01:37:57.380137 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.380107    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-5wgcd\""
Nov 04 01:37:57.380840 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.380810    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr"]
Nov 04 01:37:57.381247 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.381226    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc"]
Nov 04 01:37:57.384373 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.384358    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc"
Nov 04 01:37:57.422079 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.422049    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2s95\" (UniqueName: \"kubernetes.io/projected/9872ad2f-0534-46a9-8aa2-b23b598543d1-kube-api-access-m2s95\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr\" (UID: \"9872ad2f-0534-46a9-8aa2-b23b598543d1\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr"
Nov 04 01:37:57.422203 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.422085    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d115b55-8761-4b80-964f-0b8887372ea7-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc\" (UID: \"0d115b55-8761-4b80-964f-0b8887372ea7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc"
Nov 04 01:37:57.422249 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.422202    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9872ad2f-0534-46a9-8aa2-b23b598543d1-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr\" (UID: \"9872ad2f-0534-46a9-8aa2-b23b598543d1\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr"
Nov 04 01:37:57.422343 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.422328    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d115b55-8761-4b80-964f-0b8887372ea7-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc\" (UID: \"0d115b55-8761-4b80-964f-0b8887372ea7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc"
Nov 04 01:37:57.422379 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.422352    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j99jv\" (UniqueName: \"kubernetes.io/projected/0d115b55-8761-4b80-964f-0b8887372ea7-kube-api-access-j99jv\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc\" (UID: \"0d115b55-8761-4b80-964f-0b8887372ea7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc"
Nov 04 01:37:57.422426 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.422375    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9872ad2f-0534-46a9-8aa2-b23b598543d1-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr\" (UID: \"9872ad2f-0534-46a9-8aa2-b23b598543d1\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr"
Nov 04 01:37:57.422575 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.422558    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9872ad2f-0534-46a9-8aa2-b23b598543d1-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr\" (UID: \"9872ad2f-0534-46a9-8aa2-b23b598543d1\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr"
Nov 04 01:37:57.422654 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.422640    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9872ad2f-0534-46a9-8aa2-b23b598543d1-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr\" (UID: \"9872ad2f-0534-46a9-8aa2-b23b598543d1\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr"
Nov 04 01:37:57.435573 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.435544    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc"]
Nov 04 01:37:57.436969 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.436940    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-wwsrh\""
Nov 04 01:37:57.473065 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.473029    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh"]
Nov 04 01:37:57.476322 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.476301    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh"
Nov 04 01:37:57.484298 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.484278    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2s95\" (UniqueName: \"kubernetes.io/projected/9872ad2f-0534-46a9-8aa2-b23b598543d1-kube-api-access-m2s95\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr\" (UID: \"9872ad2f-0534-46a9-8aa2-b23b598543d1\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr"
Nov 04 01:37:57.523858 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.523827    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh\" (UID: \"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh"
Nov 04 01:37:57.523858 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.523862    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh\" (UID: \"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh"
Nov 04 01:37:57.524112 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.523895    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d115b55-8761-4b80-964f-0b8887372ea7-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc\" (UID: \"0d115b55-8761-4b80-964f-0b8887372ea7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc"
Nov 04 01:37:57.524112 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.523968    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j99jv\" (UniqueName: \"kubernetes.io/projected/0d115b55-8761-4b80-964f-0b8887372ea7-kube-api-access-j99jv\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc\" (UID: \"0d115b55-8761-4b80-964f-0b8887372ea7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc"
Nov 04 01:37:57.524112 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.524023    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dkb7\" (UniqueName: \"kubernetes.io/projected/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-kube-api-access-8dkb7\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh\" (UID: \"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh"
Nov 04 01:37:57.524112 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.524048    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d115b55-8761-4b80-964f-0b8887372ea7-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc\" (UID: \"0d115b55-8761-4b80-964f-0b8887372ea7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc"
Nov 04 01:37:57.524327 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.524284    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d115b55-8761-4b80-964f-0b8887372ea7-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc\" (UID: \"0d115b55-8761-4b80-964f-0b8887372ea7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc"
Nov 04 01:37:57.524375 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.524326    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d115b55-8761-4b80-964f-0b8887372ea7-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc\" (UID: \"0d115b55-8761-4b80-964f-0b8887372ea7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc"
Nov 04 01:37:57.535976 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.535953    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh"]
Nov 04 01:37:57.537186 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.537167    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-jzp9p\""
Nov 04 01:37:57.580825 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.577394    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr"]
Nov 04 01:37:57.582432 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.582403    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr"
Nov 04 01:37:57.584666 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.584645    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j99jv\" (UniqueName: \"kubernetes.io/projected/0d115b55-8761-4b80-964f-0b8887372ea7-kube-api-access-j99jv\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc\" (UID: \"0d115b55-8761-4b80-964f-0b8887372ea7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc"
Nov 04 01:37:57.593319 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.593296    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr"
Nov 04 01:37:57.625018 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.624986    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dkb7\" (UniqueName: \"kubernetes.io/projected/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-kube-api-access-8dkb7\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh\" (UID: \"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh"
Nov 04 01:37:57.625191 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.625043    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htgpx\" (UniqueName: \"kubernetes.io/projected/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-kube-api-access-htgpx\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr\" (UID: \"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr"
Nov 04 01:37:57.625191 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.625083    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh\" (UID: \"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh"
Nov 04 01:37:57.625191 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.625111    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh\" (UID: \"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh"
Nov 04 01:37:57.625497 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.625475    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr\" (UID: \"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr"
Nov 04 01:37:57.625592 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.625506    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr\" (UID: \"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr"
Nov 04 01:37:57.625592 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.625508    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh\" (UID: \"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh"
Nov 04 01:37:57.625592 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.625555    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh\" (UID: \"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh"
Nov 04 01:37:57.638073 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.638021    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-28wzj\""
Nov 04 01:37:57.641087 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.641059    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr"]
Nov 04 01:37:57.666145 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.665416    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8" event={"ID":"f78843c8-7092-4753-a69f-592ab02bd9b8","Type":"ContainerStarted","Data":"7fada0ad9f5c547610f406bee3950c0b42c06226e38453b2563beea2efb84f50"}
Nov 04 01:37:57.686591 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.686564    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dkb7\" (UniqueName: \"kubernetes.io/projected/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-kube-api-access-8dkb7\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh\" (UID: \"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh"
Nov 04 01:37:57.699303 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.699274    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc"
Nov 04 01:37:57.726412 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.726378    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr\" (UID: \"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr"
Nov 04 01:37:57.726590 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.726423    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr\" (UID: \"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr"
Nov 04 01:37:57.726590 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.726516    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htgpx\" (UniqueName: \"kubernetes.io/projected/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-kube-api-access-htgpx\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr\" (UID: \"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr"
Nov 04 01:37:57.726819 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.726777    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr\" (UID: \"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr"
Nov 04 01:37:57.726934 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.726849    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr\" (UID: \"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr"
Nov 04 01:37:57.787881 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.787814    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htgpx\" (UniqueName: \"kubernetes.io/projected/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-kube-api-access-htgpx\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr\" (UID: \"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr"
Nov 04 01:37:57.791556 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.791530    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh"
Nov 04 01:37:57.904225 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.904190    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr"
Nov 04 01:37:57.974417 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:57.974389    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr"]
Nov 04 01:37:57.976773 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:37:57.976745    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9872ad2f_0534_46a9_8aa2_b23b598543d1.slice/crio-edcc2f4face68e1e2de1ed4bc8031672141e6aa4590a04d03abf69c1e486d39e WatchSource:0}: Error finding container edcc2f4face68e1e2de1ed4bc8031672141e6aa4590a04d03abf69c1e486d39e: Status 404 returned error can't find the container with id edcc2f4face68e1e2de1ed4bc8031672141e6aa4590a04d03abf69c1e486d39e
Nov 04 01:37:58.082084 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:58.082004    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc"]
Nov 04 01:37:58.085558 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:37:58.085530    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d115b55_8761_4b80_964f_0b8887372ea7.slice/crio-944261843fa4e86851e92dd065a34fa72444b8b618f598e6cdfbd1bb143a5f39 WatchSource:0}: Error finding container 944261843fa4e86851e92dd065a34fa72444b8b618f598e6cdfbd1bb143a5f39: Status 404 returned error can't find the container with id 944261843fa4e86851e92dd065a34fa72444b8b618f598e6cdfbd1bb143a5f39
Nov 04 01:37:58.168409 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:58.168378    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh"]
Nov 04 01:37:58.172397 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:37:58.172354    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ff85ae8_cfb6_4cb6_8ef6_d76e57afcefb.slice/crio-8ba3e3d28091106f8644a0423b9add9131b93d51633559f808ff43528a5bc93a WatchSource:0}: Error finding container 8ba3e3d28091106f8644a0423b9add9131b93d51633559f808ff43528a5bc93a: Status 404 returned error can't find the container with id 8ba3e3d28091106f8644a0423b9add9131b93d51633559f808ff43528a5bc93a
Nov 04 01:37:58.279171 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:58.279125    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr"]
Nov 04 01:37:58.282788 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:37:58.282765    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdbe7e74_82a1_4f8d_b03c_216e0cfa4145.slice/crio-f76e98e45af7446d87e1844dd0cf28817447f875b6b3fdb23d53804f3f951803 WatchSource:0}: Error finding container f76e98e45af7446d87e1844dd0cf28817447f875b6b3fdb23d53804f3f951803: Status 404 returned error can't find the container with id f76e98e45af7446d87e1844dd0cf28817447f875b6b3fdb23d53804f3f951803
Nov 04 01:37:58.671222 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:58.671179    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh" event={"ID":"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb","Type":"ContainerStarted","Data":"8ba3e3d28091106f8644a0423b9add9131b93d51633559f808ff43528a5bc93a"}
Nov 04 01:37:58.672535 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:58.672503    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc" event={"ID":"0d115b55-8761-4b80-964f-0b8887372ea7","Type":"ContainerStarted","Data":"944261843fa4e86851e92dd065a34fa72444b8b618f598e6cdfbd1bb143a5f39"}
Nov 04 01:37:58.673865 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:58.673830    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr" event={"ID":"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145","Type":"ContainerStarted","Data":"f76e98e45af7446d87e1844dd0cf28817447f875b6b3fdb23d53804f3f951803"}
Nov 04 01:37:58.675250 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:58.675213    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr" event={"ID":"9872ad2f-0534-46a9-8aa2-b23b598543d1","Type":"ContainerStarted","Data":"edcc2f4face68e1e2de1ed4bc8031672141e6aa4590a04d03abf69c1e486d39e"}
Nov 04 01:37:59.430589 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:59.430247    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr"]
Nov 04 01:37:59.435026 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:59.434316    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr"
Nov 04 01:37:59.487926 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:59.487891    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-vzvl6\""
Nov 04 01:37:59.494380 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:59.494350    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr"]
Nov 04 01:37:59.547852 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:59.547813    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxzrx\" (UniqueName: \"kubernetes.io/projected/5d87f4e9-972a-49b4-98a1-05afb13563c0-kube-api-access-lxzrx\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr\" (UID: \"5d87f4e9-972a-49b4-98a1-05afb13563c0\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr"
Nov 04 01:37:59.548032 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:59.547873    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d87f4e9-972a-49b4-98a1-05afb13563c0-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr\" (UID: \"5d87f4e9-972a-49b4-98a1-05afb13563c0\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr"
Nov 04 01:37:59.548122 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:59.548035    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d87f4e9-972a-49b4-98a1-05afb13563c0-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr\" (UID: \"5d87f4e9-972a-49b4-98a1-05afb13563c0\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr"
Nov 04 01:37:59.648868 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:59.648830    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxzrx\" (UniqueName: \"kubernetes.io/projected/5d87f4e9-972a-49b4-98a1-05afb13563c0-kube-api-access-lxzrx\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr\" (UID: \"5d87f4e9-972a-49b4-98a1-05afb13563c0\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr"
Nov 04 01:37:59.649055 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:59.648904    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d87f4e9-972a-49b4-98a1-05afb13563c0-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr\" (UID: \"5d87f4e9-972a-49b4-98a1-05afb13563c0\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr"
Nov 04 01:37:59.649055 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:59.648996    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d87f4e9-972a-49b4-98a1-05afb13563c0-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr\" (UID: \"5d87f4e9-972a-49b4-98a1-05afb13563c0\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr"
Nov 04 01:37:59.649486 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:59.649463    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d87f4e9-972a-49b4-98a1-05afb13563c0-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr\" (UID: \"5d87f4e9-972a-49b4-98a1-05afb13563c0\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr"
Nov 04 01:37:59.650062 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:59.650039    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d87f4e9-972a-49b4-98a1-05afb13563c0-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr\" (UID: \"5d87f4e9-972a-49b4-98a1-05afb13563c0\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr"
Nov 04 01:37:59.728390 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:59.722812    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxzrx\" (UniqueName: \"kubernetes.io/projected/5d87f4e9-972a-49b4-98a1-05afb13563c0-kube-api-access-lxzrx\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr\" (UID: \"5d87f4e9-972a-49b4-98a1-05afb13563c0\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr"
Nov 04 01:37:59.754990 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:37:59.754955    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr"
Nov 04 01:38:00.174404 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:00.173036    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr"]
Nov 04 01:38:00.176611 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:38:00.176578    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d87f4e9_972a_49b4_98a1_05afb13563c0.slice/crio-9ebae908fb5c1a5d17f35a99028f06a032416a94b5da50e18b833eef769ebd49 WatchSource:0}: Error finding container 9ebae908fb5c1a5d17f35a99028f06a032416a94b5da50e18b833eef769ebd49: Status 404 returned error can't find the container with id 9ebae908fb5c1a5d17f35a99028f06a032416a94b5da50e18b833eef769ebd49
Nov 04 01:38:00.695122 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:00.694864    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr" event={"ID":"5d87f4e9-972a-49b4-98a1-05afb13563c0","Type":"ContainerStarted","Data":"9ebae908fb5c1a5d17f35a99028f06a032416a94b5da50e18b833eef769ebd49"}
Nov 04 01:38:01.738881 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.738828    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"]
Nov 04 01:38:01.743352 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.742372    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.803161 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.802982    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 04 01:38:01.804063 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.803982    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 04 01:38:01.804063 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.803989    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 04 01:38:01.836718 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.836688    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-9qgsb\""
Nov 04 01:38:01.845469 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.845315    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"]
Nov 04 01:38:01.875503 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.875470    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3775ba44-0207-42df-830d-ac56e49f6a49-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.875688 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.875528    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf7rj\" (UniqueName: \"kubernetes.io/projected/3775ba44-0207-42df-830d-ac56e49f6a49-kube-api-access-zf7rj\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.875688 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.875649    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3775ba44-0207-42df-830d-ac56e49f6a49-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.875803 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.875710    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3775ba44-0207-42df-830d-ac56e49f6a49-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.875803 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.875738    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3775ba44-0207-42df-830d-ac56e49f6a49-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.875803 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.875768    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3775ba44-0207-42df-830d-ac56e49f6a49-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.875803 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.875793    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3775ba44-0207-42df-830d-ac56e49f6a49-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.875992 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.875825    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3775ba44-0207-42df-830d-ac56e49f6a49-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.875992 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.875864    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3775ba44-0207-42df-830d-ac56e49f6a49-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.977156 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.977118    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3775ba44-0207-42df-830d-ac56e49f6a49-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.977425 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.977179    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf7rj\" (UniqueName: \"kubernetes.io/projected/3775ba44-0207-42df-830d-ac56e49f6a49-kube-api-access-zf7rj\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.977425 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.977218    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3775ba44-0207-42df-830d-ac56e49f6a49-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.977425 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.977259    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3775ba44-0207-42df-830d-ac56e49f6a49-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.977425 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.977285    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3775ba44-0207-42df-830d-ac56e49f6a49-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.977425 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.977313    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3775ba44-0207-42df-830d-ac56e49f6a49-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.977425 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.977337    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3775ba44-0207-42df-830d-ac56e49f6a49-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.977425 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.977368    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3775ba44-0207-42df-830d-ac56e49f6a49-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.977425 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.977408    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3775ba44-0207-42df-830d-ac56e49f6a49-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.979048 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.978734    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3775ba44-0207-42df-830d-ac56e49f6a49-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.979376 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.979292    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3775ba44-0207-42df-830d-ac56e49f6a49-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.980032 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.979629    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3775ba44-0207-42df-830d-ac56e49f6a49-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.980032 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.979886    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3775ba44-0207-42df-830d-ac56e49f6a49-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.980032 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.980003    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3775ba44-0207-42df-830d-ac56e49f6a49-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.983285 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.983261    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3775ba44-0207-42df-830d-ac56e49f6a49-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:01.990235 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:01.990180    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3775ba44-0207-42df-830d-ac56e49f6a49-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:02.041659 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:02.041538    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf7rj\" (UniqueName: \"kubernetes.io/projected/3775ba44-0207-42df-830d-ac56e49f6a49-kube-api-access-zf7rj\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:02.051179 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:02.051148    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3775ba44-0207-42df-830d-ac56e49f6a49-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-75b555675f-xx482\" (UID: \"3775ba44-0207-42df-830d-ac56e49f6a49\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:02.060176 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:02.059998    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:02.519498 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:02.515611    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"]
Nov 04 01:38:02.555701 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:38:02.555664    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3775ba44_0207_42df_830d_ac56e49f6a49.slice/crio-ab19be0970bd9177be83903eaa7edd6d3d49df114c279d7d28cca5068a72517f WatchSource:0}: Error finding container ab19be0970bd9177be83903eaa7edd6d3d49df114c279d7d28cca5068a72517f: Status 404 returned error can't find the container with id ab19be0970bd9177be83903eaa7edd6d3d49df114c279d7d28cca5068a72517f
Nov 04 01:38:02.715469 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:02.715413    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482" event={"ID":"3775ba44-0207-42df-830d-ac56e49f6a49","Type":"ContainerStarted","Data":"ab19be0970bd9177be83903eaa7edd6d3d49df114c279d7d28cca5068a72517f"}
Nov 04 01:38:07.750570 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:07.750118    6116 generic.go:356] "Generic (PLEG): container finished" podID="f78843c8-7092-4753-a69f-592ab02bd9b8" containerID="e7848ca7128c9384fdb074d34fafb3276fe8bd6a0fb63face5aee3411636daea" exitCode=0
Nov 04 01:38:07.750570 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:07.750240    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8" event={"ID":"f78843c8-7092-4753-a69f-592ab02bd9b8","Type":"ContainerDied","Data":"e7848ca7128c9384fdb074d34fafb3276fe8bd6a0fb63face5aee3411636daea"}
Nov 04 01:38:08.759409 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:08.758383    6116 generic.go:356] "Generic (PLEG): container finished" podID="5d87f4e9-972a-49b4-98a1-05afb13563c0" containerID="74a3b69cb87ce53085a4d509f745cae9b50e7a55a7dc82e2bed13c0d132192a2" exitCode=0
Nov 04 01:38:08.759409 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:08.758470    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr" event={"ID":"5d87f4e9-972a-49b4-98a1-05afb13563c0","Type":"ContainerDied","Data":"74a3b69cb87ce53085a4d509f745cae9b50e7a55a7dc82e2bed13c0d132192a2"}
Nov 04 01:38:09.767299 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:09.766408    6116 generic.go:356] "Generic (PLEG): container finished" podID="f78843c8-7092-4753-a69f-592ab02bd9b8" containerID="707f56bb4c8d5a07943772743bb137c29948b9b07ab7807aee39720bdc33badb" exitCode=0
Nov 04 01:38:09.767299 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:09.766476    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8" event={"ID":"f78843c8-7092-4753-a69f-592ab02bd9b8","Type":"ContainerDied","Data":"707f56bb4c8d5a07943772743bb137c29948b9b07ab7807aee39720bdc33badb"}
Nov 04 01:38:09.786034 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:09.785996    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:38:09.786144 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:09.786081    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:38:09.786144 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:09.786125    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:38:10.775768 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:10.775733    6116 generic.go:356] "Generic (PLEG): container finished" podID="9872ad2f-0534-46a9-8aa2-b23b598543d1" containerID="a53e0bc870a96315aa0de80c6f27b1ae9c2726e98ae4cc198665b7257c5bfea7" exitCode=0
Nov 04 01:38:10.776062 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:10.775839    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr" event={"ID":"9872ad2f-0534-46a9-8aa2-b23b598543d1","Type":"ContainerDied","Data":"a53e0bc870a96315aa0de80c6f27b1ae9c2726e98ae4cc198665b7257c5bfea7"}
Nov 04 01:38:10.780667 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:10.780130    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482" event={"ID":"3775ba44-0207-42df-830d-ac56e49f6a49","Type":"ContainerStarted","Data":"964872a2a3ccc8013f202664e3ca448373d7fdadb8285ca0c54684986604154e"}
Nov 04 01:38:11.033659 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:11.033600    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482" podStartSLOduration=2.808621658 podStartE2EDuration="10.033584848s" totalImagesPullingTime="7.22496319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:38:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:38:11.032571106 +0000 UTC m=+176.153848630" watchObservedRunningTime="2026-11-04 01:38:11.033584848 +0000 UTC m=+176.154862371"
Nov 04 01:38:11.061040 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:11.061006    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:11.066785 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:11.066759    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:11.786748 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:11.786700    6116 generic.go:356] "Generic (PLEG): container finished" podID="4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb" containerID="bf586030898554481de444fcd3d93be02c251b86b0f778bdad59f6e35d68c9cd" exitCode=0
Nov 04 01:38:11.787332 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:11.786789    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh" event={"ID":"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb","Type":"ContainerDied","Data":"bf586030898554481de444fcd3d93be02c251b86b0f778bdad59f6e35d68c9cd"}
Nov 04 01:38:11.789882 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:11.789856    6116 generic.go:356] "Generic (PLEG): container finished" podID="0d115b55-8761-4b80-964f-0b8887372ea7" containerID="ff0090f221eb175facfad3c380ffd8d6ae25836813ac9c6cc68a28302badd0b4" exitCode=0
Nov 04 01:38:11.790022 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:11.789928    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc" event={"ID":"0d115b55-8761-4b80-964f-0b8887372ea7","Type":"ContainerDied","Data":"ff0090f221eb175facfad3c380ffd8d6ae25836813ac9c6cc68a28302badd0b4"}
Nov 04 01:38:11.792331 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:11.792306    6116 generic.go:356] "Generic (PLEG): container finished" podID="fdbe7e74-82a1-4f8d-b03c-216e0cfa4145" containerID="28eded5227e5072f92f116382551801ce52ba0048f63552aad3b704b645f4e7c" exitCode=0
Nov 04 01:38:11.792575 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:11.792552    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr" event={"ID":"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145","Type":"ContainerDied","Data":"28eded5227e5072f92f116382551801ce52ba0048f63552aad3b704b645f4e7c"}
Nov 04 01:38:11.792647 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:11.792624    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:11.793964 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:11.793945    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482"
Nov 04 01:38:12.000325 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:38:12.000293    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:38:12.798998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:12.798902    6116 generic.go:356] "Generic (PLEG): container finished" podID="9872ad2f-0534-46a9-8aa2-b23b598543d1" containerID="c7d9798c3cb96ae486aef7b0b40dfedb42f9da243dc8b578c7073056b8ddd02c" exitCode=0
Nov 04 01:38:12.798998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:12.798980    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr" event={"ID":"9872ad2f-0534-46a9-8aa2-b23b598543d1","Type":"ContainerDied","Data":"c7d9798c3cb96ae486aef7b0b40dfedb42f9da243dc8b578c7073056b8ddd02c"}
Nov 04 01:38:12.801671 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:12.801650    6116 generic.go:356] "Generic (PLEG): container finished" podID="5d87f4e9-972a-49b4-98a1-05afb13563c0" containerID="a3a78a4fd5adf762503ba52e933e6b7995610799fe7cccc56543dd9830112b90" exitCode=0
Nov 04 01:38:12.801767 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:12.801749    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr" event={"ID":"5d87f4e9-972a-49b4-98a1-05afb13563c0","Type":"ContainerDied","Data":"a3a78a4fd5adf762503ba52e933e6b7995610799fe7cccc56543dd9830112b90"}
Nov 04 01:38:13.809568 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:13.809537    6116 generic.go:356] "Generic (PLEG): container finished" podID="4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb" containerID="36d9e83e6a55202d4b0864f42c517a9977c23698479632313212ee17feb21682" exitCode=0
Nov 04 01:38:13.810028 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:13.809619    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh" event={"ID":"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb","Type":"ContainerDied","Data":"36d9e83e6a55202d4b0864f42c517a9977c23698479632313212ee17feb21682"}
Nov 04 01:38:13.813105 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:13.813007    6116 generic.go:356] "Generic (PLEG): container finished" podID="0d115b55-8761-4b80-964f-0b8887372ea7" containerID="8c41d66fad07f7e375253877c34d2876b80a97da30c8b306d1f95050e3f2d8fe" exitCode=0
Nov 04 01:38:13.813173 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:13.813105    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc" event={"ID":"0d115b55-8761-4b80-964f-0b8887372ea7","Type":"ContainerDied","Data":"8c41d66fad07f7e375253877c34d2876b80a97da30c8b306d1f95050e3f2d8fe"}
Nov 04 01:38:14.821364 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:14.821270    6116 generic.go:356] "Generic (PLEG): container finished" podID="fdbe7e74-82a1-4f8d-b03c-216e0cfa4145" containerID="54e56ea9ac420fb1a7e32f0be94e10ae8bfd97942bb64638bdea68f720d629aa" exitCode=0
Nov 04 01:38:14.821867 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:14.821372    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr" event={"ID":"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145","Type":"ContainerDied","Data":"54e56ea9ac420fb1a7e32f0be94e10ae8bfd97942bb64638bdea68f720d629aa"}
Nov 04 01:38:19.857347 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:19.856615    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8" event={"ID":"f78843c8-7092-4753-a69f-592ab02bd9b8","Type":"ContainerStarted","Data":"9415ba58cd236eddfc26751903c16398b934f0490d031b6cce3c75bf25410db8"}
Nov 04 01:38:19.977603 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:19.977538    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8" podStartSLOduration=3.402610966 podStartE2EDuration="24.977519147s" totalImagesPullingTime="21.574908181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:37:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:38:19.976367381 +0000 UTC m=+185.097644908" watchObservedRunningTime="2026-11-04 01:38:19.977519147 +0000 UTC m=+185.098796674"
Nov 04 01:38:20.868005 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:20.867610    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr" event={"ID":"5d87f4e9-972a-49b4-98a1-05afb13563c0","Type":"ContainerStarted","Data":"fb83f37b4823ca6eba1a22c0b2267afdda311fa331143d760eb481618b9de782"}
Nov 04 01:38:20.872668 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:20.872374    6116 generic.go:356] "Generic (PLEG): container finished" podID="f78843c8-7092-4753-a69f-592ab02bd9b8" containerID="9415ba58cd236eddfc26751903c16398b934f0490d031b6cce3c75bf25410db8" exitCode=0
Nov 04 01:38:20.872668 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:20.872475    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8" event={"ID":"f78843c8-7092-4753-a69f-592ab02bd9b8","Type":"ContainerDied","Data":"9415ba58cd236eddfc26751903c16398b934f0490d031b6cce3c75bf25410db8"}
Nov 04 01:38:20.877069 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:20.877039    6116 generic.go:356] "Generic (PLEG): container finished" podID="9872ad2f-0534-46a9-8aa2-b23b598543d1" containerID="1425c938a83d4da75e623b696368a90bc30c7b3cc57650e692ccca05ec5d6547" exitCode=0
Nov 04 01:38:20.877183 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:20.877091    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr" event={"ID":"9872ad2f-0534-46a9-8aa2-b23b598543d1","Type":"ContainerDied","Data":"1425c938a83d4da75e623b696368a90bc30c7b3cc57650e692ccca05ec5d6547"}
Nov 04 01:38:21.110146 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:21.110085    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr" podStartSLOduration=3.089210634 podStartE2EDuration="22.110064566s" totalImagesPullingTime="19.020853932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:37:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:38:20.989730059 +0000 UTC m=+186.111007628" watchObservedRunningTime="2026-11-04 01:38:21.110064566 +0000 UTC m=+186.231342092"
Nov 04 01:38:21.882816 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:21.882751    6116 generic.go:356] "Generic (PLEG): container finished" podID="0d115b55-8761-4b80-964f-0b8887372ea7" containerID="009b1e65482668bdc81e95d6894922d55b8526e6c24c9379c28ec15c6673a32e" exitCode=0
Nov 04 01:38:21.883205 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:21.882824    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc" event={"ID":"0d115b55-8761-4b80-964f-0b8887372ea7","Type":"ContainerDied","Data":"009b1e65482668bdc81e95d6894922d55b8526e6c24c9379c28ec15c6673a32e"}
Nov 04 01:38:21.884996 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:21.884971    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr" event={"ID":"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145","Type":"ContainerStarted","Data":"8e173d2a0c1e9ca3bbb11c2e6cb5e8d27e46d6c9aa958dd218f7dc72e9e7abbc"}
Nov 04 01:38:21.887077 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:21.887051    6116 generic.go:356] "Generic (PLEG): container finished" podID="5d87f4e9-972a-49b4-98a1-05afb13563c0" containerID="fb83f37b4823ca6eba1a22c0b2267afdda311fa331143d760eb481618b9de782" exitCode=0
Nov 04 01:38:21.887174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:21.887120    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr" event={"ID":"5d87f4e9-972a-49b4-98a1-05afb13563c0","Type":"ContainerDied","Data":"fb83f37b4823ca6eba1a22c0b2267afdda311fa331143d760eb481618b9de782"}
Nov 04 01:38:21.888981 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:21.888958    6116 generic.go:356] "Generic (PLEG): container finished" podID="4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb" containerID="34c3757bc4e20e13373f3b023200aab4c0a36fc0c125a866c12c3caa7b4874d4" exitCode=0
Nov 04 01:38:21.889074 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:21.889027    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh" event={"ID":"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb","Type":"ContainerDied","Data":"34c3757bc4e20e13373f3b023200aab4c0a36fc0c125a866c12c3caa7b4874d4"}
Nov 04 01:38:22.011350 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.011327    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8"
Nov 04 01:38:22.026269 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.026248    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr"
Nov 04 01:38:22.125311 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.125258    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr" podStartSLOduration=3.875825108 podStartE2EDuration="25.125240739s" totalImagesPullingTime="21.249415631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:37:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:38:22.123924807 +0000 UTC m=+187.245202333" watchObservedRunningTime="2026-11-04 01:38:22.125240739 +0000 UTC m=+187.246518264"
Nov 04 01:38:22.181875 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.181841    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f78843c8-7092-4753-a69f-592ab02bd9b8-bundle\" (UniqueName: \"kubernetes.io/empty-dir/f78843c8-7092-4753-a69f-592ab02bd9b8-bundle\") pod \"f78843c8-7092-4753-a69f-592ab02bd9b8\" (UID: \"f78843c8-7092-4753-a69f-592ab02bd9b8\") "
Nov 04 01:38:22.182039 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.181880    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9872ad2f-0534-46a9-8aa2-b23b598543d1-util\" (UniqueName: \"kubernetes.io/empty-dir/9872ad2f-0534-46a9-8aa2-b23b598543d1-util\") pod \"9872ad2f-0534-46a9-8aa2-b23b598543d1\" (UID: \"9872ad2f-0534-46a9-8aa2-b23b598543d1\") "
Nov 04 01:38:22.182039 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.181928    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f78843c8-7092-4753-a69f-592ab02bd9b8-kube-api-access-s8l7d\" (UniqueName: \"kubernetes.io/projected/f78843c8-7092-4753-a69f-592ab02bd9b8-kube-api-access-s8l7d\") pod \"f78843c8-7092-4753-a69f-592ab02bd9b8\" (UID: \"f78843c8-7092-4753-a69f-592ab02bd9b8\") "
Nov 04 01:38:22.182039 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.181991    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f78843c8-7092-4753-a69f-592ab02bd9b8-util\" (UniqueName: \"kubernetes.io/empty-dir/f78843c8-7092-4753-a69f-592ab02bd9b8-util\") pod \"f78843c8-7092-4753-a69f-592ab02bd9b8\" (UID: \"f78843c8-7092-4753-a69f-592ab02bd9b8\") "
Nov 04 01:38:22.182039 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.182015    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9872ad2f-0534-46a9-8aa2-b23b598543d1-bundle\" (UniqueName: \"kubernetes.io/empty-dir/9872ad2f-0534-46a9-8aa2-b23b598543d1-bundle\") pod \"9872ad2f-0534-46a9-8aa2-b23b598543d1\" (UID: \"9872ad2f-0534-46a9-8aa2-b23b598543d1\") "
Nov 04 01:38:22.182250 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.182158    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9872ad2f-0534-46a9-8aa2-b23b598543d1-kube-api-access-m2s95\" (UniqueName: \"kubernetes.io/projected/9872ad2f-0534-46a9-8aa2-b23b598543d1-kube-api-access-m2s95\") pod \"9872ad2f-0534-46a9-8aa2-b23b598543d1\" (UID: \"9872ad2f-0534-46a9-8aa2-b23b598543d1\") "
Nov 04 01:38:22.182568 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.182543    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9872ad2f-0534-46a9-8aa2-b23b598543d1-bundle" pod "9872ad2f-0534-46a9-8aa2-b23b598543d1" (UID: "9872ad2f-0534-46a9-8aa2-b23b598543d1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:38:22.182738 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.182588    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f78843c8-7092-4753-a69f-592ab02bd9b8-bundle" pod "f78843c8-7092-4753-a69f-592ab02bd9b8" (UID: "f78843c8-7092-4753-a69f-592ab02bd9b8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:38:22.185012 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.184986    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f78843c8-7092-4753-a69f-592ab02bd9b8-kube-api-access-s8l7d" pod "f78843c8-7092-4753-a69f-592ab02bd9b8" (UID: "f78843c8-7092-4753-a69f-592ab02bd9b8"). InnerVolumeSpecName "kube-api-access-s8l7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:38:22.185012 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.184986    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9872ad2f-0534-46a9-8aa2-b23b598543d1-kube-api-access-m2s95" pod "9872ad2f-0534-46a9-8aa2-b23b598543d1" (UID: "9872ad2f-0534-46a9-8aa2-b23b598543d1"). InnerVolumeSpecName "kube-api-access-m2s95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:38:22.187892 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.187864    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f78843c8-7092-4753-a69f-592ab02bd9b8-util" pod "f78843c8-7092-4753-a69f-592ab02bd9b8" (UID: "f78843c8-7092-4753-a69f-592ab02bd9b8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:38:22.188328 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.188312    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9872ad2f-0534-46a9-8aa2-b23b598543d1-util" pod "9872ad2f-0534-46a9-8aa2-b23b598543d1" (UID: "9872ad2f-0534-46a9-8aa2-b23b598543d1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:38:22.283874 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.283838    6116 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f78843c8-7092-4753-a69f-592ab02bd9b8-util\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:22.283874 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.283869    6116 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9872ad2f-0534-46a9-8aa2-b23b598543d1-bundle\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:22.284059 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.283884    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2s95\" (UniqueName: \"kubernetes.io/projected/9872ad2f-0534-46a9-8aa2-b23b598543d1-kube-api-access-m2s95\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:22.284059 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.283897    6116 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f78843c8-7092-4753-a69f-592ab02bd9b8-bundle\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:22.284059 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.283908    6116 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9872ad2f-0534-46a9-8aa2-b23b598543d1-util\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:22.284059 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.283918    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8l7d\" (UniqueName: \"kubernetes.io/projected/f78843c8-7092-4753-a69f-592ab02bd9b8-kube-api-access-s8l7d\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:22.896778 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.896744    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8" event={"ID":"f78843c8-7092-4753-a69f-592ab02bd9b8","Type":"ContainerDied","Data":"7fada0ad9f5c547610f406bee3950c0b42c06226e38453b2563beea2efb84f50"}
Nov 04 01:38:22.897190 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.896801    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7fada0ad9f5c547610f406bee3950c0b42c06226e38453b2563beea2efb84f50"
Nov 04 01:38:22.897190 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.896757    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v7fk8"
Nov 04 01:38:22.898801 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.898778    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr"
Nov 04 01:38:22.898926 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.898798    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q57vr" event={"ID":"9872ad2f-0534-46a9-8aa2-b23b598543d1","Type":"ContainerDied","Data":"edcc2f4face68e1e2de1ed4bc8031672141e6aa4590a04d03abf69c1e486d39e"}
Nov 04 01:38:22.898926 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.898842    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="edcc2f4face68e1e2de1ed4bc8031672141e6aa4590a04d03abf69c1e486d39e"
Nov 04 01:38:22.901107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.901085    6116 generic.go:356] "Generic (PLEG): container finished" podID="fdbe7e74-82a1-4f8d-b03c-216e0cfa4145" containerID="8e173d2a0c1e9ca3bbb11c2e6cb5e8d27e46d6c9aa958dd218f7dc72e9e7abbc" exitCode=0
Nov 04 01:38:22.901213 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:22.901158    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr" event={"ID":"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145","Type":"ContainerDied","Data":"8e173d2a0c1e9ca3bbb11c2e6cb5e8d27e46d6c9aa958dd218f7dc72e9e7abbc"}
Nov 04 01:38:23.049742 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.049712    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh"
Nov 04 01:38:23.075562 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.075536    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc"
Nov 04 01:38:23.078593 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.078575    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr"
Nov 04 01:38:23.192476 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.192436    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d87f4e9-972a-49b4-98a1-05afb13563c0-kube-api-access-lxzrx\" (UniqueName: \"kubernetes.io/projected/5d87f4e9-972a-49b4-98a1-05afb13563c0-kube-api-access-lxzrx\") pod \"5d87f4e9-972a-49b4-98a1-05afb13563c0\" (UID: \"5d87f4e9-972a-49b4-98a1-05afb13563c0\") "
Nov 04 01:38:23.192658 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.192528    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d87f4e9-972a-49b4-98a1-05afb13563c0-bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d87f4e9-972a-49b4-98a1-05afb13563c0-bundle\") pod \"5d87f4e9-972a-49b4-98a1-05afb13563c0\" (UID: \"5d87f4e9-972a-49b4-98a1-05afb13563c0\") "
Nov 04 01:38:23.192658 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.192578    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-kube-api-access-8dkb7\" (UniqueName: \"kubernetes.io/projected/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-kube-api-access-8dkb7\") pod \"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb\" (UID: \"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb\") "
Nov 04 01:38:23.192658 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.192608    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d87f4e9-972a-49b4-98a1-05afb13563c0-util\" (UniqueName: \"kubernetes.io/empty-dir/5d87f4e9-972a-49b4-98a1-05afb13563c0-util\") pod \"5d87f4e9-972a-49b4-98a1-05afb13563c0\" (UID: \"5d87f4e9-972a-49b4-98a1-05afb13563c0\") "
Nov 04 01:38:23.192658 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.192644    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d115b55-8761-4b80-964f-0b8887372ea7-bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d115b55-8761-4b80-964f-0b8887372ea7-bundle\") pod \"0d115b55-8761-4b80-964f-0b8887372ea7\" (UID: \"0d115b55-8761-4b80-964f-0b8887372ea7\") "
Nov 04 01:38:23.192867 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.192670    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-bundle\") pod \"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb\" (UID: \"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb\") "
Nov 04 01:38:23.192867 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.192709    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d115b55-8761-4b80-964f-0b8887372ea7-kube-api-access-j99jv\" (UniqueName: \"kubernetes.io/projected/0d115b55-8761-4b80-964f-0b8887372ea7-kube-api-access-j99jv\") pod \"0d115b55-8761-4b80-964f-0b8887372ea7\" (UID: \"0d115b55-8761-4b80-964f-0b8887372ea7\") "
Nov 04 01:38:23.192867 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.192764    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d115b55-8761-4b80-964f-0b8887372ea7-util\" (UniqueName: \"kubernetes.io/empty-dir/0d115b55-8761-4b80-964f-0b8887372ea7-util\") pod \"0d115b55-8761-4b80-964f-0b8887372ea7\" (UID: \"0d115b55-8761-4b80-964f-0b8887372ea7\") "
Nov 04 01:38:23.192867 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.192791    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-util\" (UniqueName: \"kubernetes.io/empty-dir/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-util\") pod \"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb\" (UID: \"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb\") "
Nov 04 01:38:23.193042 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.192951    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d87f4e9-972a-49b4-98a1-05afb13563c0-bundle" pod "5d87f4e9-972a-49b4-98a1-05afb13563c0" (UID: "5d87f4e9-972a-49b4-98a1-05afb13563c0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:38:23.193097 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.193045    6116 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d87f4e9-972a-49b4-98a1-05afb13563c0-bundle\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:23.193884 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.193854    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-bundle" pod "4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb" (UID: "4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:38:23.194031 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.193918    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d115b55-8761-4b80-964f-0b8887372ea7-bundle" pod "0d115b55-8761-4b80-964f-0b8887372ea7" (UID: "0d115b55-8761-4b80-964f-0b8887372ea7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:38:23.194880 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.194855    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d87f4e9-972a-49b4-98a1-05afb13563c0-kube-api-access-lxzrx" pod "5d87f4e9-972a-49b4-98a1-05afb13563c0" (UID: "5d87f4e9-972a-49b4-98a1-05afb13563c0"). InnerVolumeSpecName "kube-api-access-lxzrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:38:23.194968 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.194895    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-kube-api-access-8dkb7" pod "4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb" (UID: "4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb"). InnerVolumeSpecName "kube-api-access-8dkb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:38:23.195166 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.195148    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d115b55-8761-4b80-964f-0b8887372ea7-kube-api-access-j99jv" pod "0d115b55-8761-4b80-964f-0b8887372ea7" (UID: "0d115b55-8761-4b80-964f-0b8887372ea7"). InnerVolumeSpecName "kube-api-access-j99jv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:38:23.198135 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.198109    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-util" pod "4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb" (UID: "4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:38:23.198738 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.198716    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d87f4e9-972a-49b4-98a1-05afb13563c0-util" pod "5d87f4e9-972a-49b4-98a1-05afb13563c0" (UID: "5d87f4e9-972a-49b4-98a1-05afb13563c0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:38:23.200684 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.200647    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d115b55-8761-4b80-964f-0b8887372ea7-util" pod "0d115b55-8761-4b80-964f-0b8887372ea7" (UID: "0d115b55-8761-4b80-964f-0b8887372ea7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:38:23.295199 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.295114    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dkb7\" (UniqueName: \"kubernetes.io/projected/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-kube-api-access-8dkb7\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:23.295393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.295379    6116 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d87f4e9-972a-49b4-98a1-05afb13563c0-util\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:23.295494 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.295483    6116 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d115b55-8761-4b80-964f-0b8887372ea7-bundle\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:23.295585 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.295576    6116 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-bundle\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:23.295656 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.295646    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j99jv\" (UniqueName: \"kubernetes.io/projected/0d115b55-8761-4b80-964f-0b8887372ea7-kube-api-access-j99jv\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:23.295726 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.295716    6116 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d115b55-8761-4b80-964f-0b8887372ea7-util\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:23.295788 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.295780    6116 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb-util\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:23.296018 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.296005    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxzrx\" (UniqueName: \"kubernetes.io/projected/5d87f4e9-972a-49b4-98a1-05afb13563c0-kube-api-access-lxzrx\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:23.907230 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.907190    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh" event={"ID":"4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb","Type":"ContainerDied","Data":"8ba3e3d28091106f8644a0423b9add9131b93d51633559f808ff43528a5bc93a"}
Nov 04 01:38:23.907230 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.907215    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vdrnh"
Nov 04 01:38:23.907733 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.907271    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ba3e3d28091106f8644a0423b9add9131b93d51633559f808ff43528a5bc93a"
Nov 04 01:38:23.909238 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.909216    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc"
Nov 04 01:38:23.909361 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.909210    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agrrhc" event={"ID":"0d115b55-8761-4b80-964f-0b8887372ea7","Type":"ContainerDied","Data":"944261843fa4e86851e92dd065a34fa72444b8b618f598e6cdfbd1bb143a5f39"}
Nov 04 01:38:23.909361 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.909343    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="944261843fa4e86851e92dd065a34fa72444b8b618f598e6cdfbd1bb143a5f39"
Nov 04 01:38:23.911216 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.911187    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr" event={"ID":"5d87f4e9-972a-49b4-98a1-05afb13563c0","Type":"ContainerDied","Data":"9ebae908fb5c1a5d17f35a99028f06a032416a94b5da50e18b833eef769ebd49"}
Nov 04 01:38:23.911342 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.911226    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ebae908fb5c1a5d17f35a99028f06a032416a94b5da50e18b833eef769ebd49"
Nov 04 01:38:23.911342 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:23.911305    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644xswjr"
Nov 04 01:38:24.026217 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:24.026197    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr"
Nov 04 01:38:24.101173 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:24.101144    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-util\" (UniqueName: \"kubernetes.io/empty-dir/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-util\") pod \"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145\" (UID: \"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145\") "
Nov 04 01:38:24.101358 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:24.101208    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-bundle\") pod \"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145\" (UID: \"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145\") "
Nov 04 01:38:24.101358 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:24.101242    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-kube-api-access-htgpx\" (UniqueName: \"kubernetes.io/projected/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-kube-api-access-htgpx\") pod \"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145\" (UID: \"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145\") "
Nov 04 01:38:24.101776 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:24.101753    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-bundle" pod "fdbe7e74-82a1-4f8d-b03c-216e0cfa4145" (UID: "fdbe7e74-82a1-4f8d-b03c-216e0cfa4145"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:38:24.103422 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:24.103398    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-kube-api-access-htgpx" pod "fdbe7e74-82a1-4f8d-b03c-216e0cfa4145" (UID: "fdbe7e74-82a1-4f8d-b03c-216e0cfa4145"). InnerVolumeSpecName "kube-api-access-htgpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:38:24.106755 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:24.106733    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-util" pod "fdbe7e74-82a1-4f8d-b03c-216e0cfa4145" (UID: "fdbe7e74-82a1-4f8d-b03c-216e0cfa4145"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:38:24.202199 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:24.202167    6116 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-util\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:24.202199 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:24.202199    6116 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-bundle\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:24.202199 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:24.202216    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htgpx\" (UniqueName: \"kubernetes.io/projected/fdbe7e74-82a1-4f8d-b03c-216e0cfa4145-kube-api-access-htgpx\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:38:24.917348 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:24.917315    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr" event={"ID":"fdbe7e74-82a1-4f8d-b03c-216e0cfa4145","Type":"ContainerDied","Data":"f76e98e45af7446d87e1844dd0cf28817447f875b6b3fdb23d53804f3f951803"}
Nov 04 01:38:24.917837 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:24.917372    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f76e98e45af7446d87e1844dd0cf28817447f875b6b3fdb23d53804f3f951803"
Nov 04 01:38:24.917837 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:24.917324    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cbb8mr"
Nov 04 01:38:27.722643 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722608    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"]
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722869    6116 cpu_manager.go:414] "containerMap: removing container" podUID="f78843c8-7092-4753-a69f-592ab02bd9b8" containerName="extract"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722879    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f78843c8-7092-4753-a69f-592ab02bd9b8" containerName="extract"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722889    6116 cpu_manager.go:414] "containerMap: removing container" podUID="0d115b55-8761-4b80-964f-0b8887372ea7" containerName="util"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722894    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d115b55-8761-4b80-964f-0b8887372ea7" containerName="util"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722906    6116 cpu_manager.go:414] "containerMap: removing container" podUID="0d115b55-8761-4b80-964f-0b8887372ea7" containerName="pull"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722912    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d115b55-8761-4b80-964f-0b8887372ea7" containerName="pull"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722920    6116 cpu_manager.go:414] "containerMap: removing container" podUID="4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb" containerName="util"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722924    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb" containerName="util"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722931    6116 cpu_manager.go:414] "containerMap: removing container" podUID="fdbe7e74-82a1-4f8d-b03c-216e0cfa4145" containerName="pull"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722935    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fdbe7e74-82a1-4f8d-b03c-216e0cfa4145" containerName="pull"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722944    6116 cpu_manager.go:414] "containerMap: removing container" podUID="9872ad2f-0534-46a9-8aa2-b23b598543d1" containerName="util"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722950    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9872ad2f-0534-46a9-8aa2-b23b598543d1" containerName="util"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722956    6116 cpu_manager.go:414] "containerMap: removing container" podUID="9872ad2f-0534-46a9-8aa2-b23b598543d1" containerName="pull"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722960    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9872ad2f-0534-46a9-8aa2-b23b598543d1" containerName="pull"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722969    6116 cpu_manager.go:414] "containerMap: removing container" podUID="4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb" containerName="pull"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722973    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb" containerName="pull"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722978    6116 cpu_manager.go:414] "containerMap: removing container" podUID="4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb" containerName="extract"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722982    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb" containerName="extract"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722990    6116 cpu_manager.go:414] "containerMap: removing container" podUID="5d87f4e9-972a-49b4-98a1-05afb13563c0" containerName="pull"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.722994    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d87f4e9-972a-49b4-98a1-05afb13563c0" containerName="pull"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723001    6116 cpu_manager.go:414] "containerMap: removing container" podUID="0d115b55-8761-4b80-964f-0b8887372ea7" containerName="extract"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723004    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d115b55-8761-4b80-964f-0b8887372ea7" containerName="extract"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723010    6116 cpu_manager.go:414] "containerMap: removing container" podUID="fdbe7e74-82a1-4f8d-b03c-216e0cfa4145" containerName="extract"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723013    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fdbe7e74-82a1-4f8d-b03c-216e0cfa4145" containerName="extract"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723020    6116 cpu_manager.go:414] "containerMap: removing container" podUID="5d87f4e9-972a-49b4-98a1-05afb13563c0" containerName="extract"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723024    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d87f4e9-972a-49b4-98a1-05afb13563c0" containerName="extract"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723030    6116 cpu_manager.go:414] "containerMap: removing container" podUID="f78843c8-7092-4753-a69f-592ab02bd9b8" containerName="util"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723034    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f78843c8-7092-4753-a69f-592ab02bd9b8" containerName="util"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723042    6116 cpu_manager.go:414] "containerMap: removing container" podUID="5d87f4e9-972a-49b4-98a1-05afb13563c0" containerName="util"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723046    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d87f4e9-972a-49b4-98a1-05afb13563c0" containerName="util"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723051    6116 cpu_manager.go:414] "containerMap: removing container" podUID="9872ad2f-0534-46a9-8aa2-b23b598543d1" containerName="extract"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723055    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9872ad2f-0534-46a9-8aa2-b23b598543d1" containerName="extract"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723062    6116 cpu_manager.go:414] "containerMap: removing container" podUID="fdbe7e74-82a1-4f8d-b03c-216e0cfa4145" containerName="util"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723066    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fdbe7e74-82a1-4f8d-b03c-216e0cfa4145" containerName="util"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723074    6116 cpu_manager.go:414] "containerMap: removing container" podUID="f78843c8-7092-4753-a69f-592ab02bd9b8" containerName="pull"
Nov 04 01:38:27.723107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723078    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f78843c8-7092-4753-a69f-592ab02bd9b8" containerName="pull"
Nov 04 01:38:27.724136 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723131    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ff85ae8-cfb6-4cb6-8ef6-d76e57afcefb" containerName="extract"
Nov 04 01:38:27.724136 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723140    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d115b55-8761-4b80-964f-0b8887372ea7" containerName="extract"
Nov 04 01:38:27.724136 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723144    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="fdbe7e74-82a1-4f8d-b03c-216e0cfa4145" containerName="extract"
Nov 04 01:38:27.724136 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723150    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d87f4e9-972a-49b4-98a1-05afb13563c0" containerName="extract"
Nov 04 01:38:27.724136 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723155    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="9872ad2f-0534-46a9-8aa2-b23b598543d1" containerName="extract"
Nov 04 01:38:27.724136 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.723161    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="f78843c8-7092-4753-a69f-592ab02bd9b8" containerName="extract"
Nov 04 01:38:27.725173 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.725152    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:27.778841 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.778809    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 04 01:38:27.778841 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.778832    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 04 01:38:27.779084 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.778872    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 04 01:38:27.779084 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.778917    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 04 01:38:27.779084 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.778988    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-xh4nj\""
Nov 04 01:38:27.779084 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.779031    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 04 01:38:27.832978 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.832943    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2ec8b208-ac97-4918-97ed-4eba50a3b562-manifests\") pod \"opendatahub-operator-controller-manager-7565df5f-548hr\" (UID: \"2ec8b208-ac97-4918-97ed-4eba50a3b562\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:27.833136 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.832990    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ec8b208-ac97-4918-97ed-4eba50a3b562-webhook-cert\") pod \"opendatahub-operator-controller-manager-7565df5f-548hr\" (UID: \"2ec8b208-ac97-4918-97ed-4eba50a3b562\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:27.833136 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.833013    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2ec8b208-ac97-4918-97ed-4eba50a3b562-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7565df5f-548hr\" (UID: \"2ec8b208-ac97-4918-97ed-4eba50a3b562\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:27.833252 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.833177    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2ec8b208-ac97-4918-97ed-4eba50a3b562-metrics-cert\") pod \"opendatahub-operator-controller-manager-7565df5f-548hr\" (UID: \"2ec8b208-ac97-4918-97ed-4eba50a3b562\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:27.833252 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.833218    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ec8b208-ac97-4918-97ed-4eba50a3b562-tmp\") pod \"opendatahub-operator-controller-manager-7565df5f-548hr\" (UID: \"2ec8b208-ac97-4918-97ed-4eba50a3b562\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:27.833357 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.833252    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drdct\" (UniqueName: \"kubernetes.io/projected/2ec8b208-ac97-4918-97ed-4eba50a3b562-kube-api-access-drdct\") pod \"opendatahub-operator-controller-manager-7565df5f-548hr\" (UID: \"2ec8b208-ac97-4918-97ed-4eba50a3b562\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:27.839041 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.839004    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"]
Nov 04 01:38:27.934287 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.934255    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2ec8b208-ac97-4918-97ed-4eba50a3b562-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7565df5f-548hr\" (UID: \"2ec8b208-ac97-4918-97ed-4eba50a3b562\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:27.934287 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.934291    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2ec8b208-ac97-4918-97ed-4eba50a3b562-metrics-cert\") pod \"opendatahub-operator-controller-manager-7565df5f-548hr\" (UID: \"2ec8b208-ac97-4918-97ed-4eba50a3b562\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:27.934612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.934308    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ec8b208-ac97-4918-97ed-4eba50a3b562-tmp\") pod \"opendatahub-operator-controller-manager-7565df5f-548hr\" (UID: \"2ec8b208-ac97-4918-97ed-4eba50a3b562\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:27.934612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.934326    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drdct\" (UniqueName: \"kubernetes.io/projected/2ec8b208-ac97-4918-97ed-4eba50a3b562-kube-api-access-drdct\") pod \"opendatahub-operator-controller-manager-7565df5f-548hr\" (UID: \"2ec8b208-ac97-4918-97ed-4eba50a3b562\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:27.934612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.934353    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2ec8b208-ac97-4918-97ed-4eba50a3b562-manifests\") pod \"opendatahub-operator-controller-manager-7565df5f-548hr\" (UID: \"2ec8b208-ac97-4918-97ed-4eba50a3b562\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:27.934612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.934389    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ec8b208-ac97-4918-97ed-4eba50a3b562-webhook-cert\") pod \"opendatahub-operator-controller-manager-7565df5f-548hr\" (UID: \"2ec8b208-ac97-4918-97ed-4eba50a3b562\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:27.934850 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.934830    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2ec8b208-ac97-4918-97ed-4eba50a3b562-manifests\") pod \"opendatahub-operator-controller-manager-7565df5f-548hr\" (UID: \"2ec8b208-ac97-4918-97ed-4eba50a3b562\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:27.936649 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.936623    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ec8b208-ac97-4918-97ed-4eba50a3b562-tmp\") pod \"opendatahub-operator-controller-manager-7565df5f-548hr\" (UID: \"2ec8b208-ac97-4918-97ed-4eba50a3b562\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:27.936907 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.936881    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2ec8b208-ac97-4918-97ed-4eba50a3b562-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7565df5f-548hr\" (UID: \"2ec8b208-ac97-4918-97ed-4eba50a3b562\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:27.937197 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.937179    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ec8b208-ac97-4918-97ed-4eba50a3b562-webhook-cert\") pod \"opendatahub-operator-controller-manager-7565df5f-548hr\" (UID: \"2ec8b208-ac97-4918-97ed-4eba50a3b562\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:27.937298 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:27.937278    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2ec8b208-ac97-4918-97ed-4eba50a3b562-metrics-cert\") pod \"opendatahub-operator-controller-manager-7565df5f-548hr\" (UID: \"2ec8b208-ac97-4918-97ed-4eba50a3b562\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:28.004152 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:28.004077    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drdct\" (UniqueName: \"kubernetes.io/projected/2ec8b208-ac97-4918-97ed-4eba50a3b562-kube-api-access-drdct\") pod \"opendatahub-operator-controller-manager-7565df5f-548hr\" (UID: \"2ec8b208-ac97-4918-97ed-4eba50a3b562\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:28.036799 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:28.036766    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:28.530070 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:28.530043    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"]
Nov 04 01:38:28.532602 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:38:28.532577    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ec8b208_ac97_4918_97ed_4eba50a3b562.slice/crio-cedc71cddf63afc7ab3431ea89b36e0b523855f59d06df1bc25b3f69665c8ad4 WatchSource:0}: Error finding container cedc71cddf63afc7ab3431ea89b36e0b523855f59d06df1bc25b3f69665c8ad4: Status 404 returned error can't find the container with id cedc71cddf63afc7ab3431ea89b36e0b523855f59d06df1bc25b3f69665c8ad4
Nov 04 01:38:28.934287 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:28.934255    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr" event={"ID":"2ec8b208-ac97-4918-97ed-4eba50a3b562","Type":"ContainerStarted","Data":"cedc71cddf63afc7ab3431ea89b36e0b523855f59d06df1bc25b3f69665c8ad4"}
Nov 04 01:38:29.661752 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:29.661714    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-jjl7l"]
Nov 04 01:38:29.663611 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:29.663596    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-jjl7l"
Nov 04 01:38:29.715775 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:29.715746    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Nov 04 01:38:29.715944 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:29.715889    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Nov 04 01:38:29.715997 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:29.715966    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-2rxp9\""
Nov 04 01:38:29.747392 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:29.747356    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3e86c818-c5ee-49ac-999a-e2f00472849c-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-jjl7l\" (UID: \"3e86c818-c5ee-49ac-999a-e2f00472849c\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-jjl7l"
Nov 04 01:38:29.747580 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:29.747413    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hchv\" (UniqueName: \"kubernetes.io/projected/3e86c818-c5ee-49ac-999a-e2f00472849c-kube-api-access-5hchv\") pod \"cert-manager-webhook-5dbbbf75cc-jjl7l\" (UID: \"3e86c818-c5ee-49ac-999a-e2f00472849c\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-jjl7l"
Nov 04 01:38:29.766189 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:29.766163    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-jjl7l"]
Nov 04 01:38:29.847884 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:29.847846    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hchv\" (UniqueName: \"kubernetes.io/projected/3e86c818-c5ee-49ac-999a-e2f00472849c-kube-api-access-5hchv\") pod \"cert-manager-webhook-5dbbbf75cc-jjl7l\" (UID: \"3e86c818-c5ee-49ac-999a-e2f00472849c\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-jjl7l"
Nov 04 01:38:29.848086 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:29.847936    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3e86c818-c5ee-49ac-999a-e2f00472849c-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-jjl7l\" (UID: \"3e86c818-c5ee-49ac-999a-e2f00472849c\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-jjl7l"
Nov 04 01:38:29.913647 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:29.913564    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3e86c818-c5ee-49ac-999a-e2f00472849c-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-jjl7l\" (UID: \"3e86c818-c5ee-49ac-999a-e2f00472849c\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-jjl7l"
Nov 04 01:38:29.913783 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:29.913669    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hchv\" (UniqueName: \"kubernetes.io/projected/3e86c818-c5ee-49ac-999a-e2f00472849c-kube-api-access-5hchv\") pod \"cert-manager-webhook-5dbbbf75cc-jjl7l\" (UID: \"3e86c818-c5ee-49ac-999a-e2f00472849c\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-jjl7l"
Nov 04 01:38:29.995996 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:29.995676    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-jjl7l"
Nov 04 01:38:30.376319 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:30.376212    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-jjl7l"]
Nov 04 01:38:30.949732 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:30.949697    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-jjl7l" event={"ID":"3e86c818-c5ee-49ac-999a-e2f00472849c","Type":"ContainerStarted","Data":"6f36774dccbb8ed400adcc05dc69203d1ab3a635abfd91b41d0d72b2e97cc3f1"}
Nov 04 01:38:32.961528 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:32.961494    6116 generic.go:356] "Generic (PLEG): container finished" podID="2ec8b208-ac97-4918-97ed-4eba50a3b562" containerID="5f355a44d96f9d2afba982d42ba19ab3cc7bdbfb1e743aad7ba35e15770468d0" exitCode=0
Nov 04 01:38:32.961992 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:32.961562    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr" event={"ID":"2ec8b208-ac97-4918-97ed-4eba50a3b562","Type":"ContainerDied","Data":"5f355a44d96f9d2afba982d42ba19ab3cc7bdbfb1e743aad7ba35e15770468d0"}
Nov 04 01:38:33.967932 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:33.967901    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr" event={"ID":"2ec8b208-ac97-4918-97ed-4eba50a3b562","Type":"ContainerStarted","Data":"b1ffe70995ce434fa5c930623cd19e17a0532319e3487d434ee69fc670b6441a"}
Nov 04 01:38:33.968365 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:33.968055    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:34.245224 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:34.245158    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr" podStartSLOduration=2.279858533 podStartE2EDuration="7.245140961s" totalImagesPullingTime="4.304672711s" totalInitContainerRuntime="660.609717ms" isStatefulPod=false podCreationTimestamp="2026-11-04 01:38:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:38:34.195557783 +0000 UTC m=+199.316835311" watchObservedRunningTime="2026-11-04 01:38:34.245140961 +0000 UTC m=+199.366418504"
Nov 04 01:38:34.973513 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:34.973479    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-jjl7l" event={"ID":"3e86c818-c5ee-49ac-999a-e2f00472849c","Type":"ContainerStarted","Data":"871662e673f0fd14bb9c475f6b4cc5fbe6abab4b62c6a9de12b914f8fc439e83"}
Nov 04 01:38:34.974117 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:34.973738    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-jjl7l"
Nov 04 01:38:35.116054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:35.116003    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-jjl7l" podStartSLOduration=2.011878941 podStartE2EDuration="6.115989131s" totalImagesPullingTime="4.10411019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:38:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:38:35.11456564 +0000 UTC m=+200.235843166" watchObservedRunningTime="2026-11-04 01:38:35.115989131 +0000 UTC m=+200.237266650"
Nov 04 01:38:35.458002 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:35.457965    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-h7sl2"]
Nov 04 01:38:35.460149 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:35.460127    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-h7sl2"
Nov 04 01:38:35.498261 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:35.498222    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjj85\" (UniqueName: \"kubernetes.io/projected/59691019-ef5b-4fea-9f18-d495d36fc536-kube-api-access-gjj85\") pod \"cert-manager-cainjector-5c57bbd4fc-h7sl2\" (UID: \"59691019-ef5b-4fea-9f18-d495d36fc536\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-h7sl2"
Nov 04 01:38:35.498429 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:35.498271    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/59691019-ef5b-4fea-9f18-d495d36fc536-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-h7sl2\" (UID: \"59691019-ef5b-4fea-9f18-d495d36fc536\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-h7sl2"
Nov 04 01:38:35.512241 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:35.512202    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-sgb4z\""
Nov 04 01:38:35.518889 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:35.518858    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-h7sl2"]
Nov 04 01:38:35.599726 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:35.599697    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjj85\" (UniqueName: \"kubernetes.io/projected/59691019-ef5b-4fea-9f18-d495d36fc536-kube-api-access-gjj85\") pod \"cert-manager-cainjector-5c57bbd4fc-h7sl2\" (UID: \"59691019-ef5b-4fea-9f18-d495d36fc536\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-h7sl2"
Nov 04 01:38:35.599726 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:35.599731    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/59691019-ef5b-4fea-9f18-d495d36fc536-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-h7sl2\" (UID: \"59691019-ef5b-4fea-9f18-d495d36fc536\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-h7sl2"
Nov 04 01:38:35.662852 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:35.662817    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/59691019-ef5b-4fea-9f18-d495d36fc536-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-h7sl2\" (UID: \"59691019-ef5b-4fea-9f18-d495d36fc536\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-h7sl2"
Nov 04 01:38:35.663017 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:35.662897    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjj85\" (UniqueName: \"kubernetes.io/projected/59691019-ef5b-4fea-9f18-d495d36fc536-kube-api-access-gjj85\") pod \"cert-manager-cainjector-5c57bbd4fc-h7sl2\" (UID: \"59691019-ef5b-4fea-9f18-d495d36fc536\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-h7sl2"
Nov 04 01:38:35.773718 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:35.773625    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-h7sl2"
Nov 04 01:38:36.146825 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:36.146739    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-h7sl2"]
Nov 04 01:38:36.149631 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:38:36.149605    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59691019_ef5b_4fea_9f18_d495d36fc536.slice/crio-c8b77a3419873e7ca5a06f973a82e676b2dae398fd8060f4b97b1d68c69f2659 WatchSource:0}: Error finding container c8b77a3419873e7ca5a06f973a82e676b2dae398fd8060f4b97b1d68c69f2659: Status 404 returned error can't find the container with id c8b77a3419873e7ca5a06f973a82e676b2dae398fd8060f4b97b1d68c69f2659
Nov 04 01:38:36.983583 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:36.983543    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-h7sl2" event={"ID":"59691019-ef5b-4fea-9f18-d495d36fc536","Type":"ContainerStarted","Data":"7b64a9e04d2c134ad57095cd62793639ef126b94c396c11fb3e78042702b13ad"}
Nov 04 01:38:36.983781 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:36.983595    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-h7sl2" event={"ID":"59691019-ef5b-4fea-9f18-d495d36fc536","Type":"ContainerStarted","Data":"c8b77a3419873e7ca5a06f973a82e676b2dae398fd8060f4b97b1d68c69f2659"}
Nov 04 01:38:37.098006 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:37.097955    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-h7sl2" podStartSLOduration=2.097940792 podStartE2EDuration="2.097940792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:38:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:38:37.096779899 +0000 UTC m=+202.218057425" watchObservedRunningTime="2026-11-04 01:38:37.097940792 +0000 UTC m=+202.219218317"
Nov 04 01:38:38.392094 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:38.392061    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-6lccj"]
Nov 04 01:38:38.398401 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:38.398383    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6lccj"
Nov 04 01:38:38.424741 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:38.424709    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m85lm\" (UniqueName: \"kubernetes.io/projected/5e4062f4-e3c4-43aa-9b38-ca3d3585bfba-kube-api-access-m85lm\") pod \"jobset-operator-674cd59dc5-6lccj\" (UID: \"5e4062f4-e3c4-43aa-9b38-ca3d3585bfba\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6lccj"
Nov 04 01:38:38.424857 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:38.424752    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e4062f4-e3c4-43aa-9b38-ca3d3585bfba-tmp\") pod \"jobset-operator-674cd59dc5-6lccj\" (UID: \"5e4062f4-e3c4-43aa-9b38-ca3d3585bfba\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6lccj"
Nov 04 01:38:38.454103 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:38.454076    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-6lccj"]
Nov 04 01:38:38.502354 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:38.502323    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:38:38.502532 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:38.502368    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 04 01:38:38.502532 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:38.502427    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-lr5rv\""
Nov 04 01:38:38.526065 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:38.526036    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m85lm\" (UniqueName: \"kubernetes.io/projected/5e4062f4-e3c4-43aa-9b38-ca3d3585bfba-kube-api-access-m85lm\") pod \"jobset-operator-674cd59dc5-6lccj\" (UID: \"5e4062f4-e3c4-43aa-9b38-ca3d3585bfba\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6lccj"
Nov 04 01:38:38.526226 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:38.526071    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e4062f4-e3c4-43aa-9b38-ca3d3585bfba-tmp\") pod \"jobset-operator-674cd59dc5-6lccj\" (UID: \"5e4062f4-e3c4-43aa-9b38-ca3d3585bfba\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6lccj"
Nov 04 01:38:38.526430 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:38.526414    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e4062f4-e3c4-43aa-9b38-ca3d3585bfba-tmp\") pod \"jobset-operator-674cd59dc5-6lccj\" (UID: \"5e4062f4-e3c4-43aa-9b38-ca3d3585bfba\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6lccj"
Nov 04 01:38:38.592397 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:38.592360    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m85lm\" (UniqueName: \"kubernetes.io/projected/5e4062f4-e3c4-43aa-9b38-ca3d3585bfba-kube-api-access-m85lm\") pod \"jobset-operator-674cd59dc5-6lccj\" (UID: \"5e4062f4-e3c4-43aa-9b38-ca3d3585bfba\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6lccj"
Nov 04 01:38:38.709567 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:38.709533    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6lccj"
Nov 04 01:38:39.091179 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:38:39.091146    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e4062f4_e3c4_43aa_9b38_ca3d3585bfba.slice/crio-d1abba2b0ca12f210e2eac3615c61261cecee0d8e441349a2ae6ef4eb827b9e6 WatchSource:0}: Error finding container d1abba2b0ca12f210e2eac3615c61261cecee0d8e441349a2ae6ef4eb827b9e6: Status 404 returned error can't find the container with id d1abba2b0ca12f210e2eac3615c61261cecee0d8e441349a2ae6ef4eb827b9e6
Nov 04 01:38:39.104636 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:39.104609    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-6lccj"]
Nov 04 01:38:39.998601 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:39.998565    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6lccj" event={"ID":"5e4062f4-e3c4-43aa-9b38-ca3d3585bfba","Type":"ContainerStarted","Data":"d1abba2b0ca12f210e2eac3615c61261cecee0d8e441349a2ae6ef4eb827b9e6"}
Nov 04 01:38:40.980162 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:40.980132    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-jjl7l"
Nov 04 01:38:41.995050 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:38:41.995013    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:38:43.013498 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:43.013442    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6lccj" event={"ID":"5e4062f4-e3c4-43aa-9b38-ca3d3585bfba","Type":"ContainerStarted","Data":"3f2bf33e26b52bc531be1dc3f1c34b3f2f71c427cb9b6125aa8b9ee7e109e616"}
Nov 04 01:38:43.134159 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:43.134079    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6lccj" podStartSLOduration=2.110133403 podStartE2EDuration="5.134063963s" totalImagesPullingTime="3.02393056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:38:43.133784449 +0000 UTC m=+208.255061978" watchObservedRunningTime="2026-11-04 01:38:43.134063963 +0000 UTC m=+208.255341470"
Nov 04 01:38:44.976010 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:44.975981    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr"
Nov 04 01:38:53.468953 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:53.468919    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-jt25s"]
Nov 04 01:38:53.471090 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:53.471070    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jt25s"
Nov 04 01:38:53.567635 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:53.567597    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv4sf\" (UniqueName: \"kubernetes.io/projected/451d239a-b06e-4ca3-af9b-7affd25cb5ff-kube-api-access-rv4sf\") pod \"limitador-operator-controller-manager-5778bb4449-jt25s\" (UID: \"451d239a-b06e-4ca3-af9b-7affd25cb5ff\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jt25s"
Nov 04 01:38:53.573235 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:53.573216    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 04 01:38:53.573324 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:53.573223    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 04 01:38:53.573324 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:53.573237    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-kgs2b\""
Nov 04 01:38:53.574793 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:53.574774    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-jt25s"]
Nov 04 01:38:53.668351 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:53.668311    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv4sf\" (UniqueName: \"kubernetes.io/projected/451d239a-b06e-4ca3-af9b-7affd25cb5ff-kube-api-access-rv4sf\") pod \"limitador-operator-controller-manager-5778bb4449-jt25s\" (UID: \"451d239a-b06e-4ca3-af9b-7affd25cb5ff\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jt25s"
Nov 04 01:38:53.731127 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:53.731059    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv4sf\" (UniqueName: \"kubernetes.io/projected/451d239a-b06e-4ca3-af9b-7affd25cb5ff-kube-api-access-rv4sf\") pod \"limitador-operator-controller-manager-5778bb4449-jt25s\" (UID: \"451d239a-b06e-4ca3-af9b-7affd25cb5ff\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jt25s"
Nov 04 01:38:53.782025 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:53.781986    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jt25s"
Nov 04 01:38:54.150776 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:54.150746    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-jt25s"]
Nov 04 01:38:54.153193 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:38:54.153160    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod451d239a_b06e_4ca3_af9b_7affd25cb5ff.slice/crio-8389e62be7d91aaf02cf347ffb273be0567b82dcd1b92f170d467aed26378439 WatchSource:0}: Error finding container 8389e62be7d91aaf02cf347ffb273be0567b82dcd1b92f170d467aed26378439: Status 404 returned error can't find the container with id 8389e62be7d91aaf02cf347ffb273be0567b82dcd1b92f170d467aed26378439
Nov 04 01:38:55.064658 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:55.064620    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jt25s" event={"ID":"451d239a-b06e-4ca3-af9b-7affd25cb5ff","Type":"ContainerStarted","Data":"8389e62be7d91aaf02cf347ffb273be0567b82dcd1b92f170d467aed26378439"}
Nov 04 01:38:58.080087 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:58.080053    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jt25s" event={"ID":"451d239a-b06e-4ca3-af9b-7affd25cb5ff","Type":"ContainerStarted","Data":"c4f34c18ed983c832493f9ba93ae3676e2afbc1195f3790185d23ba348333e3c"}
Nov 04 01:38:58.080540 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:58.080160    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jt25s"
Nov 04 01:38:58.200355 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:38:58.200300    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jt25s" podStartSLOduration=1.716238583 podStartE2EDuration="5.200284313s" totalImagesPullingTime="3.48404573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:38:58.198847934 +0000 UTC m=+223.320125451" watchObservedRunningTime="2026-11-04 01:38:58.200284313 +0000 UTC m=+223.321561818"
Nov 04 01:39:07.528978 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:07.528938    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-84fbf4d7ff-zlfjk"]
Nov 04 01:39:07.595357 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:07.595334    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-84fbf4d7ff-zlfjk"
Nov 04 01:39:07.608535 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:07.608507    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-84fbf4d7ff-zlfjk"]
Nov 04 01:39:07.673716 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:07.673671    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 04 01:39:07.697430 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:07.697396    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 04 01:39:07.697430 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:07.697396    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-4c68c\""
Nov 04 01:39:07.791081 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:07.790996    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/599691e1-af11-4605-b63b-3dd597be5801-tmp\") pod \"kube-auth-proxy-84fbf4d7ff-zlfjk\" (UID: \"599691e1-af11-4605-b63b-3dd597be5801\") " pod="openshift-ingress/kube-auth-proxy-84fbf4d7ff-zlfjk"
Nov 04 01:39:07.791081 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:07.791040    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/599691e1-af11-4605-b63b-3dd597be5801-tls-certs\") pod \"kube-auth-proxy-84fbf4d7ff-zlfjk\" (UID: \"599691e1-af11-4605-b63b-3dd597be5801\") " pod="openshift-ingress/kube-auth-proxy-84fbf4d7ff-zlfjk"
Nov 04 01:39:07.791081 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:07.791058    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvq84\" (UniqueName: \"kubernetes.io/projected/599691e1-af11-4605-b63b-3dd597be5801-kube-api-access-wvq84\") pod \"kube-auth-proxy-84fbf4d7ff-zlfjk\" (UID: \"599691e1-af11-4605-b63b-3dd597be5801\") " pod="openshift-ingress/kube-auth-proxy-84fbf4d7ff-zlfjk"
Nov 04 01:39:07.892163 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:07.892127    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/599691e1-af11-4605-b63b-3dd597be5801-tmp\") pod \"kube-auth-proxy-84fbf4d7ff-zlfjk\" (UID: \"599691e1-af11-4605-b63b-3dd597be5801\") " pod="openshift-ingress/kube-auth-proxy-84fbf4d7ff-zlfjk"
Nov 04 01:39:07.892319 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:07.892173    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/599691e1-af11-4605-b63b-3dd597be5801-tls-certs\") pod \"kube-auth-proxy-84fbf4d7ff-zlfjk\" (UID: \"599691e1-af11-4605-b63b-3dd597be5801\") " pod="openshift-ingress/kube-auth-proxy-84fbf4d7ff-zlfjk"
Nov 04 01:39:07.892319 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:07.892261    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvq84\" (UniqueName: \"kubernetes.io/projected/599691e1-af11-4605-b63b-3dd597be5801-kube-api-access-wvq84\") pod \"kube-auth-proxy-84fbf4d7ff-zlfjk\" (UID: \"599691e1-af11-4605-b63b-3dd597be5801\") " pod="openshift-ingress/kube-auth-proxy-84fbf4d7ff-zlfjk"
Nov 04 01:39:07.894673 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:07.894642    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/599691e1-af11-4605-b63b-3dd597be5801-tmp\") pod \"kube-auth-proxy-84fbf4d7ff-zlfjk\" (UID: \"599691e1-af11-4605-b63b-3dd597be5801\") " pod="openshift-ingress/kube-auth-proxy-84fbf4d7ff-zlfjk"
Nov 04 01:39:07.894799 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:07.894767    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/599691e1-af11-4605-b63b-3dd597be5801-tls-certs\") pod \"kube-auth-proxy-84fbf4d7ff-zlfjk\" (UID: \"599691e1-af11-4605-b63b-3dd597be5801\") " pod="openshift-ingress/kube-auth-proxy-84fbf4d7ff-zlfjk"
Nov 04 01:39:07.953054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:07.953021    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvq84\" (UniqueName: \"kubernetes.io/projected/599691e1-af11-4605-b63b-3dd597be5801-kube-api-access-wvq84\") pod \"kube-auth-proxy-84fbf4d7ff-zlfjk\" (UID: \"599691e1-af11-4605-b63b-3dd597be5801\") " pod="openshift-ingress/kube-auth-proxy-84fbf4d7ff-zlfjk"
Nov 04 01:39:08.157851 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.157821    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"]
Nov 04 01:39:08.178775 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.178749    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.223324 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.223292    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-84fbf4d7ff-zlfjk"
Nov 04 01:39:08.231360 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.231330    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"]
Nov 04 01:39:08.291246 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.291219    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-jnp2f\""
Nov 04 01:39:08.295861 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.295399    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66431dd6-1363-4794-972e-9d274b31dfc1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.295861 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.295481    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66431dd6-1363-4794-972e-9d274b31dfc1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.295861 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.295517    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66431dd6-1363-4794-972e-9d274b31dfc1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.295861 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.295554    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66431dd6-1363-4794-972e-9d274b31dfc1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.295861 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.295579    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66431dd6-1363-4794-972e-9d274b31dfc1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.295861 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.295669    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66431dd6-1363-4794-972e-9d274b31dfc1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.295861 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.295708    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66431dd6-1363-4794-972e-9d274b31dfc1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.295861 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.295753    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66431dd6-1363-4794-972e-9d274b31dfc1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.295861 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.295816    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j6xh\" (UniqueName: \"kubernetes.io/projected/66431dd6-1363-4794-972e-9d274b31dfc1-kube-api-access-6j6xh\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.396556 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.396519    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66431dd6-1363-4794-972e-9d274b31dfc1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.396738 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.396574    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66431dd6-1363-4794-972e-9d274b31dfc1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.396738 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.396598    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66431dd6-1363-4794-972e-9d274b31dfc1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.396738 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.396617    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66431dd6-1363-4794-972e-9d274b31dfc1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.396738 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.396657    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66431dd6-1363-4794-972e-9d274b31dfc1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.396738 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.396691    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66431dd6-1363-4794-972e-9d274b31dfc1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.396738 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.396733    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66431dd6-1363-4794-972e-9d274b31dfc1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.397029 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.396766    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j6xh\" (UniqueName: \"kubernetes.io/projected/66431dd6-1363-4794-972e-9d274b31dfc1-kube-api-access-6j6xh\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.397029 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.396791    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66431dd6-1363-4794-972e-9d274b31dfc1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.397133 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.397060    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66431dd6-1363-4794-972e-9d274b31dfc1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.397697 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.397401    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66431dd6-1363-4794-972e-9d274b31dfc1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.397697 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.397401    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66431dd6-1363-4794-972e-9d274b31dfc1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.397697 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.397524    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66431dd6-1363-4794-972e-9d274b31dfc1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.397697 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.397651    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66431dd6-1363-4794-972e-9d274b31dfc1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.399208 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.399182    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66431dd6-1363-4794-972e-9d274b31dfc1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.400099 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.400076    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66431dd6-1363-4794-972e-9d274b31dfc1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.468318 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.468232    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j6xh\" (UniqueName: \"kubernetes.io/projected/66431dd6-1363-4794-972e-9d274b31dfc1-kube-api-access-6j6xh\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.469030 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.469012    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66431dd6-1363-4794-972e-9d274b31dfc1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-58c98c4485dftg7\" (UID: \"66431dd6-1363-4794-972e-9d274b31dfc1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.515313 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.515278    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:08.619414 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.619373    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-84fbf4d7ff-zlfjk"]
Nov 04 01:39:08.623216 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:39:08.623185    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod599691e1_af11_4605_b63b_3dd597be5801.slice/crio-6fdb24ffdc7c890598ceb86e8ee79f77a6b22453c9cd172e4601f6583a2077a6 WatchSource:0}: Error finding container 6fdb24ffdc7c890598ceb86e8ee79f77a6b22453c9cd172e4601f6583a2077a6: Status 404 returned error can't find the container with id 6fdb24ffdc7c890598ceb86e8ee79f77a6b22453c9cd172e4601f6583a2077a6
Nov 04 01:39:08.931018 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.930996    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"]
Nov 04 01:39:08.934336 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:39:08.934293    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66431dd6_1363_4794_972e_9d274b31dfc1.slice/crio-5b7177df76047a5d24753ec4b686aa1118e24498203f0fd3d47e66843f1b49ce WatchSource:0}: Error finding container 5b7177df76047a5d24753ec4b686aa1118e24498203f0fd3d47e66843f1b49ce: Status 404 returned error can't find the container with id 5b7177df76047a5d24753ec4b686aa1118e24498203f0fd3d47e66843f1b49ce
Nov 04 01:39:08.936711 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.936679    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:39:08.936814 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.936783    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:39:08.936895 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:08.936829    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:39:09.086182 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:09.086153    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-jt25s"
Nov 04 01:39:09.125942 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:09.125908    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-84fbf4d7ff-zlfjk" event={"ID":"599691e1-af11-4605-b63b-3dd597be5801","Type":"ContainerStarted","Data":"6fdb24ffdc7c890598ceb86e8ee79f77a6b22453c9cd172e4601f6583a2077a6"}
Nov 04 01:39:09.127368 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:09.127344    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7" event={"ID":"66431dd6-1363-4794-972e-9d274b31dfc1","Type":"ContainerStarted","Data":"f3106555d848792f7a6dc59111e42b7c9e59a6d5a7cec2b1106574daf317e13e"}
Nov 04 01:39:09.127531 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:09.127391    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7" event={"ID":"66431dd6-1363-4794-972e-9d274b31dfc1","Type":"ContainerStarted","Data":"5b7177df76047a5d24753ec4b686aa1118e24498203f0fd3d47e66843f1b49ce"}
Nov 04 01:39:09.336639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:09.336543    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7" podStartSLOduration=1.336525075 podStartE2EDuration="1.336525075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:39:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:39:09.335815086 +0000 UTC m=+234.457092624" watchObservedRunningTime="2026-11-04 01:39:09.336525075 +0000 UTC m=+234.457802598"
Nov 04 01:39:09.515874 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:09.515845    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:10.521895 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:10.521859    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:11.138071 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:11.138012    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:11.138914 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:11.138891    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7"
Nov 04 01:39:11.991531 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:39:11.991498    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:39:12.142890 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:12.142852    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-84fbf4d7ff-zlfjk" event={"ID":"599691e1-af11-4605-b63b-3dd597be5801","Type":"ContainerStarted","Data":"948b00c4097dde6774ce430ca14b969f32e99a19dec5e5c5ece843d4509f13e5"}
Nov 04 01:39:12.261814 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:12.261707    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-84fbf4d7ff-zlfjk" podStartSLOduration=2.4769077680000002 podStartE2EDuration="5.261688528s" totalImagesPullingTime="2.78478076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:39:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:39:12.260549107 +0000 UTC m=+237.381826632" watchObservedRunningTime="2026-11-04 01:39:12.261688528 +0000 UTC m=+237.382966055"
Nov 04 01:39:13.843582 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:13.843544    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:39:13.845857 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:13.845836    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"
Nov 04 01:39:13.898257 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:13.898232    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5hchf\""
Nov 04 01:39:13.947761 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:13.947729    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 04 01:39:13.949415 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:13.949384    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx5xv\" (UniqueName: \"kubernetes.io/projected/66494911-a785-4301-9401-c5ba28818edb-kube-api-access-mx5xv\") pod \"limitador-limitador-7c9b87d574-mzk5s\" (UID: \"66494911-a785-4301-9401-c5ba28818edb\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"
Nov 04 01:39:13.949572 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:13.949472    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/66494911-a785-4301-9401-c5ba28818edb-config-file\") pod \"limitador-limitador-7c9b87d574-mzk5s\" (UID: \"66494911-a785-4301-9401-c5ba28818edb\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"
Nov 04 01:39:13.949572 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:13.949540    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:39:14.050704 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:14.050664    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx5xv\" (UniqueName: \"kubernetes.io/projected/66494911-a785-4301-9401-c5ba28818edb-kube-api-access-mx5xv\") pod \"limitador-limitador-7c9b87d574-mzk5s\" (UID: \"66494911-a785-4301-9401-c5ba28818edb\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"
Nov 04 01:39:14.050892 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:14.050726    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/66494911-a785-4301-9401-c5ba28818edb-config-file\") pod \"limitador-limitador-7c9b87d574-mzk5s\" (UID: \"66494911-a785-4301-9401-c5ba28818edb\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"
Nov 04 01:39:14.051345 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:14.051318    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/66494911-a785-4301-9401-c5ba28818edb-config-file\") pod \"limitador-limitador-7c9b87d574-mzk5s\" (UID: \"66494911-a785-4301-9401-c5ba28818edb\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"
Nov 04 01:39:14.111342 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:14.111273    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx5xv\" (UniqueName: \"kubernetes.io/projected/66494911-a785-4301-9401-c5ba28818edb-kube-api-access-mx5xv\") pod \"limitador-limitador-7c9b87d574-mzk5s\" (UID: \"66494911-a785-4301-9401-c5ba28818edb\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"
Nov 04 01:39:14.156656 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:14.156629    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"
Nov 04 01:39:14.167043 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:14.167018    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:39:14.532934 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:14.532907    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:39:14.535326 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:39:14.535298    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66494911_a785_4301_9401_c5ba28818edb.slice/crio-40204286893f5abef5bd5d5c9cba2ccd643c6423ab833d6621636e5748c9d2d3 WatchSource:0}: Error finding container 40204286893f5abef5bd5d5c9cba2ccd643c6423ab833d6621636e5748c9d2d3: Status 404 returned error can't find the container with id 40204286893f5abef5bd5d5c9cba2ccd643c6423ab833d6621636e5748c9d2d3
Nov 04 01:39:15.157033 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:15.156999    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-mzk5s" event={"ID":"66494911-a785-4301-9401-c5ba28818edb","Type":"ContainerStarted","Data":"40204286893f5abef5bd5d5c9cba2ccd643c6423ab833d6621636e5748c9d2d3"}
Nov 04 01:39:17.167158 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:17.167120    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-mzk5s" event={"ID":"66494911-a785-4301-9401-c5ba28818edb","Type":"ContainerStarted","Data":"cf6ca01626376b835031570383b9b69509ee237545f1cdfd0f32aee72dd7970d"}
Nov 04 01:39:17.167574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:17.167317    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"
Nov 04 01:39:17.285044 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:17.284980    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-mzk5s" podStartSLOduration=2.653276922 podStartE2EDuration="4.284960892s" totalImagesPullingTime="1.63168397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:39:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:39:17.284097169 +0000 UTC m=+242.405374717" watchObservedRunningTime="2026-11-04 01:39:17.284960892 +0000 UTC m=+242.406238421"
Nov 04 01:39:28.172956 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:39:28.172927    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"
Nov 04 01:39:41.990956 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:39:41.990916    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:40:06.849294 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:06.849263    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"]
Nov 04 01:40:06.852718 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:06.852696    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"
Nov 04 01:40:06.895079 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:06.895045    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d6ffa44d-084d-4df2-962c-d1f8cc30507e-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4qznm\" (UID: \"d6ffa44d-084d-4df2-962c-d1f8cc30507e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"
Nov 04 01:40:06.895250 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:06.895116    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ffa44d-084d-4df2-962c-d1f8cc30507e-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4qznm\" (UID: \"d6ffa44d-084d-4df2-962c-d1f8cc30507e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"
Nov 04 01:40:06.895250 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:06.895208    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/d6ffa44d-084d-4df2-962c-d1f8cc30507e-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4qznm\" (UID: \"d6ffa44d-084d-4df2-962c-d1f8cc30507e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"
Nov 04 01:40:06.895250 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:06.895240    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwdfw\" (UniqueName: \"kubernetes.io/projected/d6ffa44d-084d-4df2-962c-d1f8cc30507e-kube-api-access-dwdfw\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4qznm\" (UID: \"d6ffa44d-084d-4df2-962c-d1f8cc30507e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"
Nov 04 01:40:06.904840 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:06.904817    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 04 01:40:06.904987 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:06.904972    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Nov 04 01:40:06.905044 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:06.904978    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Nov 04 01:40:06.954753 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:06.954717    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Nov 04 01:40:06.954957 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:06.954767    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 04 01:40:06.954957 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:06.954933    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-zh2bz\""
Nov 04 01:40:06.956363 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:06.956344    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"]
Nov 04 01:40:06.995990 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:06.995954    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ffa44d-084d-4df2-962c-d1f8cc30507e-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4qznm\" (UID: \"d6ffa44d-084d-4df2-962c-d1f8cc30507e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"
Nov 04 01:40:06.996189 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:06.996005    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/d6ffa44d-084d-4df2-962c-d1f8cc30507e-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4qznm\" (UID: \"d6ffa44d-084d-4df2-962c-d1f8cc30507e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"
Nov 04 01:40:06.996189 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:06.996029    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwdfw\" (UniqueName: \"kubernetes.io/projected/d6ffa44d-084d-4df2-962c-d1f8cc30507e-kube-api-access-dwdfw\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4qznm\" (UID: \"d6ffa44d-084d-4df2-962c-d1f8cc30507e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"
Nov 04 01:40:06.996189 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:06.996079    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d6ffa44d-084d-4df2-962c-d1f8cc30507e-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4qznm\" (UID: \"d6ffa44d-084d-4df2-962c-d1f8cc30507e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"
Nov 04 01:40:06.996189 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:40:06.996098    6116 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found
Nov 04 01:40:06.996404 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:40:06.996209    6116 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d6ffa44d-084d-4df2-962c-d1f8cc30507e-proxy-source podName:d6ffa44d-084d-4df2-962c-d1f8cc30507e nodeName:}" failed. No retries permitted until 2026-11-04 01:40:07.496185763 +0000 UTC m=+292.617463290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/d6ffa44d-084d-4df2-962c-d1f8cc30507e-proxy-source") pod "usage-logs-tenancy-proxy-9c5d49bcd-4qznm" (UID: "d6ffa44d-084d-4df2-962c-d1f8cc30507e") : configmap "usage-tenancy-proxy-source" not found
Nov 04 01:40:06.996404 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:40:06.996245    6116 secret.go:189] Couldn't get secret opendatahub/usage-logs-tenancy-proxy-tls: secret "usage-logs-tenancy-proxy-tls" not found
Nov 04 01:40:06.996404 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:40:06.996295    6116 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6ffa44d-084d-4df2-962c-d1f8cc30507e-tls-cert podName:d6ffa44d-084d-4df2-962c-d1f8cc30507e nodeName:}" failed. No retries permitted until 2026-11-04 01:40:07.496282774 +0000 UTC m=+292.617560293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-cert" (UniqueName: "kubernetes.io/secret/d6ffa44d-084d-4df2-962c-d1f8cc30507e-tls-cert") pod "usage-logs-tenancy-proxy-9c5d49bcd-4qznm" (UID: "d6ffa44d-084d-4df2-962c-d1f8cc30507e") : secret "usage-logs-tenancy-proxy-tls" not found
Nov 04 01:40:06.996787 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:06.996764    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ffa44d-084d-4df2-962c-d1f8cc30507e-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4qznm\" (UID: \"d6ffa44d-084d-4df2-962c-d1f8cc30507e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"
Nov 04 01:40:07.059101 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:07.059070    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwdfw\" (UniqueName: \"kubernetes.io/projected/d6ffa44d-084d-4df2-962c-d1f8cc30507e-kube-api-access-dwdfw\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4qznm\" (UID: \"d6ffa44d-084d-4df2-962c-d1f8cc30507e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"
Nov 04 01:40:07.500367 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:07.500330    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/d6ffa44d-084d-4df2-962c-d1f8cc30507e-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4qznm\" (UID: \"d6ffa44d-084d-4df2-962c-d1f8cc30507e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"
Nov 04 01:40:07.500573 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:07.500386    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d6ffa44d-084d-4df2-962c-d1f8cc30507e-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4qznm\" (UID: \"d6ffa44d-084d-4df2-962c-d1f8cc30507e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"
Nov 04 01:40:07.501219 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:07.501185    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/d6ffa44d-084d-4df2-962c-d1f8cc30507e-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4qznm\" (UID: \"d6ffa44d-084d-4df2-962c-d1f8cc30507e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"
Nov 04 01:40:07.502823 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:07.502798    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d6ffa44d-084d-4df2-962c-d1f8cc30507e-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4qznm\" (UID: \"d6ffa44d-084d-4df2-962c-d1f8cc30507e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"
Nov 04 01:40:07.765069 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:07.764989    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"
Nov 04 01:40:08.137841 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:08.137792    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"]
Nov 04 01:40:08.140604 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:40:08.140576    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6ffa44d_084d_4df2_962c_d1f8cc30507e.slice/crio-e54a409825540d2a562e7c838fefb20cb1ea22f1dbb483984f3c4b080e78b1e1 WatchSource:0}: Error finding container e54a409825540d2a562e7c838fefb20cb1ea22f1dbb483984f3c4b080e78b1e1: Status 404 returned error can't find the container with id e54a409825540d2a562e7c838fefb20cb1ea22f1dbb483984f3c4b080e78b1e1
Nov 04 01:40:08.387791 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:08.387708    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm" event={"ID":"d6ffa44d-084d-4df2-962c-d1f8cc30507e","Type":"ContainerStarted","Data":"e54a409825540d2a562e7c838fefb20cb1ea22f1dbb483984f3c4b080e78b1e1"}
Nov 04 01:40:12.004981 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:40:12.004911    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:40:15.044423 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:15.044399    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/prometheus/0.log"
Nov 04 01:40:15.045252 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:15.045227    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/prometheus/0.log"
Nov 04 01:40:15.077406 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:15.077157    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6lfbv_192e2fec-fefb-4bc5-9516-ed7464963a10/ovn-acl-logging/0.log"
Nov 04 01:40:15.078582 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:15.078500    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6lfbv_192e2fec-fefb-4bc5-9516-ed7464963a10/ovn-acl-logging/0.log"
Nov 04 01:40:21.454420 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:21.454384    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm" event={"ID":"d6ffa44d-084d-4df2-962c-d1f8cc30507e","Type":"ContainerStarted","Data":"c8ded91527374f47201e790e63b2679e50609eb805f4a15905e1b1fd0b11f1ab"}
Nov 04 01:40:21.454883 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:21.454510    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"
Nov 04 01:40:21.571463 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:21.571394    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm" podStartSLOduration=2.3552630580000002 podStartE2EDuration="15.571380147s" totalImagesPullingTime="13.216117089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:40:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:40:21.57009341 +0000 UTC m=+306.691370937" watchObservedRunningTime="2026-11-04 01:40:21.571380147 +0000 UTC m=+306.692657669"
Nov 04 01:40:32.476290 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:32.476260    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm"
Nov 04 01:40:41.393531 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.393494    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-lgdkj"]
Nov 04 01:40:41.399021 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.399003    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-lgdkj"
Nov 04 01:40:41.459009 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.458976    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-lgdkj"]
Nov 04 01:40:41.506730 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.506698    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kftfz\" (UniqueName: \"kubernetes.io/projected/74baa758-c88e-4814-926d-a59456774d5f-kube-api-access-kftfz\") pod \"model-serving-api-75bd89fdfc-lgdkj\" (UID: \"74baa758-c88e-4814-926d-a59456774d5f\") " pod="opendatahub/model-serving-api-75bd89fdfc-lgdkj"
Nov 04 01:40:41.506923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.506766    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74baa758-c88e-4814-926d-a59456774d5f-tls-certs\") pod \"model-serving-api-75bd89fdfc-lgdkj\" (UID: \"74baa758-c88e-4814-926d-a59456774d5f\") " pod="opendatahub/model-serving-api-75bd89fdfc-lgdkj"
Nov 04 01:40:41.507009 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.506990    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-dbzp7\""
Nov 04 01:40:41.507044 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.506998    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Nov 04 01:40:41.508513 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.508495    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-l5vpc"]
Nov 04 01:40:41.511931 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.511917    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-l5vpc"
Nov 04 01:40:41.564820 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.564792    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-l5vpc"]
Nov 04 01:40:41.565012 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.564859    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-rpcl4\""
Nov 04 01:40:41.565012 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.564916    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Nov 04 01:40:41.565012 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.565011    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Nov 04 01:40:41.607815 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.607783    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kftfz\" (UniqueName: \"kubernetes.io/projected/74baa758-c88e-4814-926d-a59456774d5f-kube-api-access-kftfz\") pod \"model-serving-api-75bd89fdfc-lgdkj\" (UID: \"74baa758-c88e-4814-926d-a59456774d5f\") " pod="opendatahub/model-serving-api-75bd89fdfc-lgdkj"
Nov 04 01:40:41.607994 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.607841    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74baa758-c88e-4814-926d-a59456774d5f-tls-certs\") pod \"model-serving-api-75bd89fdfc-lgdkj\" (UID: \"74baa758-c88e-4814-926d-a59456774d5f\") " pod="opendatahub/model-serving-api-75bd89fdfc-lgdkj"
Nov 04 01:40:41.607994 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.607871    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61e56398-a014-42ad-89e9-dbd7fa34ec59-metrics-certs\") pod \"odh-model-controller-769875db56-l5vpc\" (UID: \"61e56398-a014-42ad-89e9-dbd7fa34ec59\") " pod="opendatahub/odh-model-controller-769875db56-l5vpc"
Nov 04 01:40:41.607994 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.607902    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61e56398-a014-42ad-89e9-dbd7fa34ec59-cert\") pod \"odh-model-controller-769875db56-l5vpc\" (UID: \"61e56398-a014-42ad-89e9-dbd7fa34ec59\") " pod="opendatahub/odh-model-controller-769875db56-l5vpc"
Nov 04 01:40:41.607994 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.607923    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbmmj\" (UniqueName: \"kubernetes.io/projected/61e56398-a014-42ad-89e9-dbd7fa34ec59-kube-api-access-rbmmj\") pod \"odh-model-controller-769875db56-l5vpc\" (UID: \"61e56398-a014-42ad-89e9-dbd7fa34ec59\") " pod="opendatahub/odh-model-controller-769875db56-l5vpc"
Nov 04 01:40:41.610367 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.610339    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74baa758-c88e-4814-926d-a59456774d5f-tls-certs\") pod \"model-serving-api-75bd89fdfc-lgdkj\" (UID: \"74baa758-c88e-4814-926d-a59456774d5f\") " pod="opendatahub/model-serving-api-75bd89fdfc-lgdkj"
Nov 04 01:40:41.673908 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.673876    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kftfz\" (UniqueName: \"kubernetes.io/projected/74baa758-c88e-4814-926d-a59456774d5f-kube-api-access-kftfz\") pod \"model-serving-api-75bd89fdfc-lgdkj\" (UID: \"74baa758-c88e-4814-926d-a59456774d5f\") " pod="opendatahub/model-serving-api-75bd89fdfc-lgdkj"
Nov 04 01:40:41.708601 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.708567    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61e56398-a014-42ad-89e9-dbd7fa34ec59-metrics-certs\") pod \"odh-model-controller-769875db56-l5vpc\" (UID: \"61e56398-a014-42ad-89e9-dbd7fa34ec59\") " pod="opendatahub/odh-model-controller-769875db56-l5vpc"
Nov 04 01:40:41.708769 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.708607    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61e56398-a014-42ad-89e9-dbd7fa34ec59-cert\") pod \"odh-model-controller-769875db56-l5vpc\" (UID: \"61e56398-a014-42ad-89e9-dbd7fa34ec59\") " pod="opendatahub/odh-model-controller-769875db56-l5vpc"
Nov 04 01:40:41.708769 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.708625    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbmmj\" (UniqueName: \"kubernetes.io/projected/61e56398-a014-42ad-89e9-dbd7fa34ec59-kube-api-access-rbmmj\") pod \"odh-model-controller-769875db56-l5vpc\" (UID: \"61e56398-a014-42ad-89e9-dbd7fa34ec59\") " pod="opendatahub/odh-model-controller-769875db56-l5vpc"
Nov 04 01:40:41.711210 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.711191    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-lgdkj"
Nov 04 01:40:41.711731 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.711712    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61e56398-a014-42ad-89e9-dbd7fa34ec59-cert\") pod \"odh-model-controller-769875db56-l5vpc\" (UID: \"61e56398-a014-42ad-89e9-dbd7fa34ec59\") " pod="opendatahub/odh-model-controller-769875db56-l5vpc"
Nov 04 01:40:41.711797 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.711751    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/61e56398-a014-42ad-89e9-dbd7fa34ec59-metrics-certs\") pod \"odh-model-controller-769875db56-l5vpc\" (UID: \"61e56398-a014-42ad-89e9-dbd7fa34ec59\") " pod="opendatahub/odh-model-controller-769875db56-l5vpc"
Nov 04 01:40:41.769393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.769360    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbmmj\" (UniqueName: \"kubernetes.io/projected/61e56398-a014-42ad-89e9-dbd7fa34ec59-kube-api-access-rbmmj\") pod \"odh-model-controller-769875db56-l5vpc\" (UID: \"61e56398-a014-42ad-89e9-dbd7fa34ec59\") " pod="opendatahub/odh-model-controller-769875db56-l5vpc"
Nov 04 01:40:41.823989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:41.823952    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-l5vpc"
Nov 04 01:40:41.990605 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:40:41.990520    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:40:42.087675 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:42.087647    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-lgdkj"]
Nov 04 01:40:42.089347 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:40:42.089312    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74baa758_c88e_4814_926d_a59456774d5f.slice/crio-d9d772aacf184e23a41b0c72e5da0e84f4ed6ca1591f6be1c4a620cdf94d41f6 WatchSource:0}: Error finding container d9d772aacf184e23a41b0c72e5da0e84f4ed6ca1591f6be1c4a620cdf94d41f6: Status 404 returned error can't find the container with id d9d772aacf184e23a41b0c72e5da0e84f4ed6ca1591f6be1c4a620cdf94d41f6
Nov 04 01:40:42.091216 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:42.091200    6116 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:40:42.192122 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:42.192098    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-l5vpc"]
Nov 04 01:40:42.194480 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:40:42.194439    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61e56398_a014_42ad_89e9_dbd7fa34ec59.slice/crio-ef86dcd1c98c226abd2ea3fde5dc81d8e438e7973b3f567b3cc01567a1517aa0 WatchSource:0}: Error finding container ef86dcd1c98c226abd2ea3fde5dc81d8e438e7973b3f567b3cc01567a1517aa0: Status 404 returned error can't find the container with id ef86dcd1c98c226abd2ea3fde5dc81d8e438e7973b3f567b3cc01567a1517aa0
Nov 04 01:40:42.556916 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:42.556878    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-lgdkj" event={"ID":"74baa758-c88e-4814-926d-a59456774d5f","Type":"ContainerStarted","Data":"d9d772aacf184e23a41b0c72e5da0e84f4ed6ca1591f6be1c4a620cdf94d41f6"}
Nov 04 01:40:42.557961 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:42.557940    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-l5vpc" event={"ID":"61e56398-a014-42ad-89e9-dbd7fa34ec59","Type":"ContainerStarted","Data":"ef86dcd1c98c226abd2ea3fde5dc81d8e438e7973b3f567b3cc01567a1517aa0"}
Nov 04 01:40:45.580039 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:45.580001    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-lgdkj" event={"ID":"74baa758-c88e-4814-926d-a59456774d5f","Type":"ContainerStarted","Data":"55df2083a4b79fc0de8e404fc9c02b23559ab3967135015ae2c7fe34cc8130f3"}
Nov 04 01:40:45.580501 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:45.580095    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-lgdkj"
Nov 04 01:40:45.696391 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:45.696334    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-lgdkj" podStartSLOduration=1.433245048 podStartE2EDuration="4.696320631s" totalImagesPullingTime="3.263075583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:40:45.694810265 +0000 UTC m=+330.816087792" watchObservedRunningTime="2026-11-04 01:40:45.696320631 +0000 UTC m=+330.817598156"
Nov 04 01:40:46.586412 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:46.586325    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-l5vpc" event={"ID":"61e56398-a014-42ad-89e9-dbd7fa34ec59","Type":"ContainerStarted","Data":"12a29f3ef401e9c5c797e0059e9db87f24c4dbad45ab4c9b42adc495ec2a9049"}
Nov 04 01:40:46.586412 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:46.586406    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-l5vpc"
Nov 04 01:40:46.703163 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:46.703110    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-l5vpc" podStartSLOduration=1.5707240709999999 podStartE2EDuration="5.703094592s" totalImagesPullingTime="4.132370521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:40:46.701485527 +0000 UTC m=+331.822763050" watchObservedRunningTime="2026-11-04 01:40:46.703094592 +0000 UTC m=+331.824372119"
Nov 04 01:40:49.881900 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:49.881862    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:40:56.591127 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:56.591094    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-lgdkj"
Nov 04 01:40:57.593347 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:40:57.593320    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-l5vpc"
Nov 04 01:41:11.990691 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:41:11.990648    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:41:41.990617 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:41:41.990577    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:42:11.990756 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:42:11.990721    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:42:36.151681 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:36.151644    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-qkgcd"]
Nov 04 01:42:36.155155 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:36.155139    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d8dddb676-qkgcd"
Nov 04 01:42:36.201740 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:36.201696    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7-metrics-certs\") pod \"ai-gateway-controller-5d8dddb676-qkgcd\" (UID: \"4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-qkgcd"
Nov 04 01:42:36.201917 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:36.201837    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x98ts\" (UniqueName: \"kubernetes.io/projected/4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7-kube-api-access-x98ts\") pod \"ai-gateway-controller-5d8dddb676-qkgcd\" (UID: \"4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-qkgcd"
Nov 04 01:42:36.256705 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:36.256658    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-ncvcg\""
Nov 04 01:42:36.256705 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:36.256665    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Nov 04 01:42:36.257007 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:36.256986    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Nov 04 01:42:36.258168 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:36.258148    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-qkgcd"]
Nov 04 01:42:36.303290 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:36.303209    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7-metrics-certs\") pod \"ai-gateway-controller-5d8dddb676-qkgcd\" (UID: \"4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-qkgcd"
Nov 04 01:42:36.303290 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:36.303268    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x98ts\" (UniqueName: \"kubernetes.io/projected/4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7-kube-api-access-x98ts\") pod \"ai-gateway-controller-5d8dddb676-qkgcd\" (UID: \"4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-qkgcd"
Nov 04 01:42:36.305744 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:36.305716    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7-metrics-certs\") pod \"ai-gateway-controller-5d8dddb676-qkgcd\" (UID: \"4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-qkgcd"
Nov 04 01:42:36.365826 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:36.365791    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x98ts\" (UniqueName: \"kubernetes.io/projected/4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7-kube-api-access-x98ts\") pod \"ai-gateway-controller-5d8dddb676-qkgcd\" (UID: \"4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7\") " pod="opendatahub/ai-gateway-controller-5d8dddb676-qkgcd"
Nov 04 01:42:36.467015 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:36.466977    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d8dddb676-qkgcd"
Nov 04 01:42:36.835809 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:36.835652    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-qkgcd"]
Nov 04 01:42:36.838540 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:42:36.838514    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e2b32cb_3af4_4cd6_a8a1_a84b0f796fc7.slice/crio-913bfe17a6da73904867fd7f1945ac7c77f964ccbc05f6e2e0a9519001e439a6 WatchSource:0}: Error finding container 913bfe17a6da73904867fd7f1945ac7c77f964ccbc05f6e2e0a9519001e439a6: Status 404 returned error can't find the container with id 913bfe17a6da73904867fd7f1945ac7c77f964ccbc05f6e2e0a9519001e439a6
Nov 04 01:42:37.079818 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:37.079783    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d8dddb676-qkgcd" event={"ID":"4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7","Type":"ContainerStarted","Data":"913bfe17a6da73904867fd7f1945ac7c77f964ccbc05f6e2e0a9519001e439a6"}
Nov 04 01:42:41.099049 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:41.099012    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d8dddb676-qkgcd" event={"ID":"4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7","Type":"ContainerStarted","Data":"72a5b368ea3f4bec7d07420b8b1ab1f04941def5f2331e2c4d7b5db78319e37d"}
Nov 04 01:42:41.099431 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:41.099089    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-5d8dddb676-qkgcd"
Nov 04 01:42:41.214548 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:41.214478    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-5d8dddb676-qkgcd" podStartSLOduration=1.9186312110000001 podStartE2EDuration="5.214439686s" totalImagesPullingTime="3.295808475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:42:41.214374434 +0000 UTC m=+446.335651963" watchObservedRunningTime="2026-11-04 01:42:41.214439686 +0000 UTC m=+446.335717213"
Nov 04 01:42:41.990324 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:42:41.990291    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:42:44.472994 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.472951    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-8ndrn"]
Nov 04 01:42:44.476432 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.476410    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-8ndrn"
Nov 04 01:42:44.528634 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.528607    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Nov 04 01:42:44.578177 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.578136    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-z9sg6\""
Nov 04 01:42:44.579761 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.579732    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-8ndrn"]
Nov 04 01:42:44.583520 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.583495    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2fa444a-b78e-4836-bc3e-61cb35f69987-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-8ndrn\" (UID: \"e2fa444a-b78e-4836-bc3e-61cb35f69987\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-8ndrn"
Nov 04 01:42:44.583634 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.583549    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw9wm\" (UniqueName: \"kubernetes.io/projected/e2fa444a-b78e-4836-bc3e-61cb35f69987-kube-api-access-xw9wm\") pod \"payload-pre-processing-7b98bb678c-8ndrn\" (UID: \"e2fa444a-b78e-4836-bc3e-61cb35f69987\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-8ndrn"
Nov 04 01:42:44.614710 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.614672    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-tv42d"]
Nov 04 01:42:44.618161 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.618143    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bb9489676-tv42d"
Nov 04 01:42:44.676335 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.676272    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-tv42d"]
Nov 04 01:42:44.684313 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.684281    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg7t8\" (UniqueName: \"kubernetes.io/projected/91242f71-8f7b-4743-a636-1a45d4fbcf89-kube-api-access-mg7t8\") pod \"payload-processing-7bb9489676-tv42d\" (UID: \"91242f71-8f7b-4743-a636-1a45d4fbcf89\") " pod="openshift-ingress/payload-processing-7bb9489676-tv42d"
Nov 04 01:42:44.684489 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.684323    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2fa444a-b78e-4836-bc3e-61cb35f69987-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-8ndrn\" (UID: \"e2fa444a-b78e-4836-bc3e-61cb35f69987\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-8ndrn"
Nov 04 01:42:44.684489 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.684349    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91242f71-8f7b-4743-a636-1a45d4fbcf89-plugins-config-volume\") pod \"payload-processing-7bb9489676-tv42d\" (UID: \"91242f71-8f7b-4743-a636-1a45d4fbcf89\") " pod="openshift-ingress/payload-processing-7bb9489676-tv42d"
Nov 04 01:42:44.684604 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.684585    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw9wm\" (UniqueName: \"kubernetes.io/projected/e2fa444a-b78e-4836-bc3e-61cb35f69987-kube-api-access-xw9wm\") pod \"payload-pre-processing-7b98bb678c-8ndrn\" (UID: \"e2fa444a-b78e-4836-bc3e-61cb35f69987\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-8ndrn"
Nov 04 01:42:44.685378 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.685358    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2fa444a-b78e-4836-bc3e-61cb35f69987-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-8ndrn\" (UID: \"e2fa444a-b78e-4836-bc3e-61cb35f69987\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-8ndrn"
Nov 04 01:42:44.745249 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.745177    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw9wm\" (UniqueName: \"kubernetes.io/projected/e2fa444a-b78e-4836-bc3e-61cb35f69987-kube-api-access-xw9wm\") pod \"payload-pre-processing-7b98bb678c-8ndrn\" (UID: \"e2fa444a-b78e-4836-bc3e-61cb35f69987\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-8ndrn"
Nov 04 01:42:44.785987 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.785950    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91242f71-8f7b-4743-a636-1a45d4fbcf89-plugins-config-volume\") pod \"payload-processing-7bb9489676-tv42d\" (UID: \"91242f71-8f7b-4743-a636-1a45d4fbcf89\") " pod="openshift-ingress/payload-processing-7bb9489676-tv42d"
Nov 04 01:42:44.786157 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.786020    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg7t8\" (UniqueName: \"kubernetes.io/projected/91242f71-8f7b-4743-a636-1a45d4fbcf89-kube-api-access-mg7t8\") pod \"payload-processing-7bb9489676-tv42d\" (UID: \"91242f71-8f7b-4743-a636-1a45d4fbcf89\") " pod="openshift-ingress/payload-processing-7bb9489676-tv42d"
Nov 04 01:42:44.786684 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.786658    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91242f71-8f7b-4743-a636-1a45d4fbcf89-plugins-config-volume\") pod \"payload-processing-7bb9489676-tv42d\" (UID: \"91242f71-8f7b-4743-a636-1a45d4fbcf89\") " pod="openshift-ingress/payload-processing-7bb9489676-tv42d"
Nov 04 01:42:44.788871 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.788855    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-8ndrn"
Nov 04 01:42:44.855584 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.852249    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg7t8\" (UniqueName: \"kubernetes.io/projected/91242f71-8f7b-4743-a636-1a45d4fbcf89-kube-api-access-mg7t8\") pod \"payload-processing-7bb9489676-tv42d\" (UID: \"91242f71-8f7b-4743-a636-1a45d4fbcf89\") " pod="openshift-ingress/payload-processing-7bb9489676-tv42d"
Nov 04 01:42:44.930758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:44.930718    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bb9489676-tv42d"
Nov 04 01:42:45.157830 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:45.157806    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-8ndrn"]
Nov 04 01:42:45.159866 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:42:45.159831    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2fa444a_b78e_4836_bc3e_61cb35f69987.slice/crio-84721ac149e218328fc1c4c5d4b4415d123b787eef062b071377b302d509650e WatchSource:0}: Error finding container 84721ac149e218328fc1c4c5d4b4415d123b787eef062b071377b302d509650e: Status 404 returned error can't find the container with id 84721ac149e218328fc1c4c5d4b4415d123b787eef062b071377b302d509650e
Nov 04 01:42:45.300916 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:45.300892    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-tv42d"]
Nov 04 01:42:45.302883 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:42:45.302851    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91242f71_8f7b_4743_a636_1a45d4fbcf89.slice/crio-9b2d4a584526ad0ca7778116c6749c094985e71813d9bbe1fbc7f69c6ed75a8d WatchSource:0}: Error finding container 9b2d4a584526ad0ca7778116c6749c094985e71813d9bbe1fbc7f69c6ed75a8d: Status 404 returned error can't find the container with id 9b2d4a584526ad0ca7778116c6749c094985e71813d9bbe1fbc7f69c6ed75a8d
Nov 04 01:42:46.122577 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:46.122529    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bb9489676-tv42d" event={"ID":"91242f71-8f7b-4743-a636-1a45d4fbcf89","Type":"ContainerStarted","Data":"9b2d4a584526ad0ca7778116c6749c094985e71813d9bbe1fbc7f69c6ed75a8d"}
Nov 04 01:42:46.124100 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:46.124063    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-8ndrn" event={"ID":"e2fa444a-b78e-4836-bc3e-61cb35f69987","Type":"ContainerStarted","Data":"84721ac149e218328fc1c4c5d4b4415d123b787eef062b071377b302d509650e"}
Nov 04 01:42:48.136497 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:48.136433    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bb9489676-tv42d" event={"ID":"91242f71-8f7b-4743-a636-1a45d4fbcf89","Type":"ContainerStarted","Data":"9b67ec3f7702d55426e7b24676d033121dbd7f79b1a0f596ce0af55211dd39d2"}
Nov 04 01:42:48.136972 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:48.136548    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bb9489676-tv42d"
Nov 04 01:42:48.137913 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:48.137891    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-8ndrn" event={"ID":"e2fa444a-b78e-4836-bc3e-61cb35f69987","Type":"ContainerStarted","Data":"7cb13334e5983ad626bff3520692158c17ceb59b0577efbf803dbc09dfd88742"}
Nov 04 01:42:48.138046 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:48.138030    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-8ndrn"
Nov 04 01:42:48.252929 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:48.252875    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7bb9489676-tv42d" podStartSLOduration=1.51149241 podStartE2EDuration="4.252860774s" totalImagesPullingTime="2.741368364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:42:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:42:48.251205891 +0000 UTC m=+453.372483416" watchObservedRunningTime="2026-11-04 01:42:48.252860774 +0000 UTC m=+453.374138299"
Nov 04 01:42:48.367810 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:48.367755    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-8ndrn" podStartSLOduration=1.6368041070000001 podStartE2EDuration="4.367739533s" totalImagesPullingTime="2.730935426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:42:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:42:48.366321714 +0000 UTC m=+453.487599258" watchObservedRunningTime="2026-11-04 01:42:48.367739533 +0000 UTC m=+453.489017058"
Nov 04 01:42:52.105467 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:52.105421    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-5d8dddb676-qkgcd"
Nov 04 01:42:52.478277 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:52.478245    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-6d4cf6d5f-9wgr6"]
Nov 04 01:42:52.481790 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:52.481765    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6d4cf6d5f-9wgr6"
Nov 04 01:42:52.540601 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:52.540566    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-6d4cf6d5f-9wgr6"]
Nov 04 01:42:52.656799 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:52.656758    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwr5g\" (UniqueName: \"kubernetes.io/projected/e62dea92-10fa-4569-9477-2010e4897b58-kube-api-access-vwr5g\") pod \"ai-gateway-controller-6d4cf6d5f-9wgr6\" (UID: \"e62dea92-10fa-4569-9477-2010e4897b58\") " pod="opendatahub/ai-gateway-controller-6d4cf6d5f-9wgr6"
Nov 04 01:42:52.656981 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:52.656808    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e62dea92-10fa-4569-9477-2010e4897b58-metrics-certs\") pod \"ai-gateway-controller-6d4cf6d5f-9wgr6\" (UID: \"e62dea92-10fa-4569-9477-2010e4897b58\") " pod="opendatahub/ai-gateway-controller-6d4cf6d5f-9wgr6"
Nov 04 01:42:52.757872 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:52.757782    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e62dea92-10fa-4569-9477-2010e4897b58-metrics-certs\") pod \"ai-gateway-controller-6d4cf6d5f-9wgr6\" (UID: \"e62dea92-10fa-4569-9477-2010e4897b58\") " pod="opendatahub/ai-gateway-controller-6d4cf6d5f-9wgr6"
Nov 04 01:42:52.758016 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:52.757908    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwr5g\" (UniqueName: \"kubernetes.io/projected/e62dea92-10fa-4569-9477-2010e4897b58-kube-api-access-vwr5g\") pod \"ai-gateway-controller-6d4cf6d5f-9wgr6\" (UID: \"e62dea92-10fa-4569-9477-2010e4897b58\") " pod="opendatahub/ai-gateway-controller-6d4cf6d5f-9wgr6"
Nov 04 01:42:52.760413 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:52.760381    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e62dea92-10fa-4569-9477-2010e4897b58-metrics-certs\") pod \"ai-gateway-controller-6d4cf6d5f-9wgr6\" (UID: \"e62dea92-10fa-4569-9477-2010e4897b58\") " pod="opendatahub/ai-gateway-controller-6d4cf6d5f-9wgr6"
Nov 04 01:42:52.819380 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:52.819340    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwr5g\" (UniqueName: \"kubernetes.io/projected/e62dea92-10fa-4569-9477-2010e4897b58-kube-api-access-vwr5g\") pod \"ai-gateway-controller-6d4cf6d5f-9wgr6\" (UID: \"e62dea92-10fa-4569-9477-2010e4897b58\") " pod="opendatahub/ai-gateway-controller-6d4cf6d5f-9wgr6"
Nov 04 01:42:53.094632 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:53.094542    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6d4cf6d5f-9wgr6"
Nov 04 01:42:53.463991 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:53.463961    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-6d4cf6d5f-9wgr6"]
Nov 04 01:42:53.470877 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:42:53.470850    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode62dea92_10fa_4569_9477_2010e4897b58.slice/crio-34128503bafbc3462139c6f250937d5cbaadf18f90f514761dc6c613362c628f WatchSource:0}: Error finding container 34128503bafbc3462139c6f250937d5cbaadf18f90f514761dc6c613362c628f: Status 404 returned error can't find the container with id 34128503bafbc3462139c6f250937d5cbaadf18f90f514761dc6c613362c628f
Nov 04 01:42:54.170700 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:54.170659    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6d4cf6d5f-9wgr6" event={"ID":"e62dea92-10fa-4569-9477-2010e4897b58","Type":"ContainerStarted","Data":"34128503bafbc3462139c6f250937d5cbaadf18f90f514761dc6c613362c628f"}
Nov 04 01:42:55.176314 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:55.176278    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6d4cf6d5f-9wgr6" event={"ID":"e62dea92-10fa-4569-9477-2010e4897b58","Type":"ContainerStarted","Data":"50152eacfbf8d0dde83fe3e63a8289c26d554f1517a343580be4c6a78efbdfe5"}
Nov 04 01:42:55.176726 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:55.176357    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-6d4cf6d5f-9wgr6"
Nov 04 01:42:55.293489 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:55.293420    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-6d4cf6d5f-9wgr6" podStartSLOduration=2.337916993 podStartE2EDuration="3.293406964s" totalImagesPullingTime="955.489971ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:42:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:42:55.291704061 +0000 UTC m=+460.412981587" watchObservedRunningTime="2026-11-04 01:42:55.293406964 +0000 UTC m=+460.414684483"
Nov 04 01:42:59.145049 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:59.145019    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-8ndrn"
Nov 04 01:42:59.145568 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:42:59.145174    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bb9489676-tv42d"
Nov 04 01:43:06.182976 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:06.182947    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-6d4cf6d5f-9wgr6"
Nov 04 01:43:06.328544 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:06.328501    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-qkgcd"]
Nov 04 01:43:06.328751 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:06.328717    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-5d8dddb676-qkgcd" podUID="4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7" containerName="manager" containerID="cri-o://72a5b368ea3f4bec7d07420b8b1ab1f04941def5f2331e2c4d7b5db78319e37d" gracePeriod=30
Nov 04 01:43:06.570838 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:06.570802    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d8dddb676-qkgcd"
Nov 04 01:43:06.584027 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:06.584002    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7-metrics-certs\") pod \"4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7\" (UID: \"4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7\") "
Nov 04 01:43:06.584150 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:06.584106    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7-kube-api-access-x98ts\" (UniqueName: \"kubernetes.io/projected/4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7-kube-api-access-x98ts\") pod \"4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7\" (UID: \"4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7\") "
Nov 04 01:43:06.586382 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:06.586349    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7-metrics-certs" pod "4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7" (UID: "4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:43:06.586560 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:06.586538    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7-kube-api-access-x98ts" pod "4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7" (UID: "4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7"). InnerVolumeSpecName "kube-api-access-x98ts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:43:06.685068 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:06.685029    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x98ts\" (UniqueName: \"kubernetes.io/projected/4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7-kube-api-access-x98ts\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:43:06.685068 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:06.685066    6116 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7-metrics-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:43:07.236295 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:07.236261    6116 generic.go:356] "Generic (PLEG): container finished" podID="4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7" containerID="72a5b368ea3f4bec7d07420b8b1ab1f04941def5f2331e2c4d7b5db78319e37d" exitCode=0
Nov 04 01:43:07.236818 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:07.236316    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d8dddb676-qkgcd"
Nov 04 01:43:07.236818 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:07.236339    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d8dddb676-qkgcd" event={"ID":"4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7","Type":"ContainerDied","Data":"72a5b368ea3f4bec7d07420b8b1ab1f04941def5f2331e2c4d7b5db78319e37d"}
Nov 04 01:43:07.236818 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:07.236376    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d8dddb676-qkgcd" event={"ID":"4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7","Type":"ContainerDied","Data":"913bfe17a6da73904867fd7f1945ac7c77f964ccbc05f6e2e0a9519001e439a6"}
Nov 04 01:43:07.236818 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:07.236397    6116 scope.go:122] "RemoveContainer" containerID="72a5b368ea3f4bec7d07420b8b1ab1f04941def5f2331e2c4d7b5db78319e37d"
Nov 04 01:43:07.245058 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:07.245039    6116 scope.go:122] "RemoveContainer" containerID="72a5b368ea3f4bec7d07420b8b1ab1f04941def5f2331e2c4d7b5db78319e37d"
Nov 04 01:43:07.245345 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:43:07.245327    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72a5b368ea3f4bec7d07420b8b1ab1f04941def5f2331e2c4d7b5db78319e37d\": container with ID starting with 72a5b368ea3f4bec7d07420b8b1ab1f04941def5f2331e2c4d7b5db78319e37d not found: ID does not exist" containerID="72a5b368ea3f4bec7d07420b8b1ab1f04941def5f2331e2c4d7b5db78319e37d"
Nov 04 01:43:07.245389 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:07.245358    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72a5b368ea3f4bec7d07420b8b1ab1f04941def5f2331e2c4d7b5db78319e37d"} err="failed to get container status \"72a5b368ea3f4bec7d07420b8b1ab1f04941def5f2331e2c4d7b5db78319e37d\": rpc error: code = NotFound desc = could not find container \"72a5b368ea3f4bec7d07420b8b1ab1f04941def5f2331e2c4d7b5db78319e37d\": container with ID starting with 72a5b368ea3f4bec7d07420b8b1ab1f04941def5f2331e2c4d7b5db78319e37d not found: ID does not exist"
Nov 04 01:43:07.408094 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:07.408066    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-qkgcd"]
Nov 04 01:43:07.411424 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:07.411403    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-5d8dddb676-qkgcd"]
Nov 04 01:43:09.026567 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:09.026537    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7" path="/var/lib/kubelet/pods/4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7/volumes"
Nov 04 01:43:11.991567 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:43:11.991531    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:43:16.908605 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:16.908568    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"]
Nov 04 01:43:16.908991 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:16.908963    6116 cpu_manager.go:414] "containerMap: removing container" podUID="4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7" containerName="manager"
Nov 04 01:43:16.908991 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:16.908975    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7" containerName="manager"
Nov 04 01:43:16.909064 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:16.909036    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e2b32cb-3af4-4cd6-a8a1-a84b0f796fc7" containerName="manager"
Nov 04 01:43:16.914884 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:16.914858    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:16.968491 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:16.968441    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Nov 04 01:43:16.974899 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:16.974864    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f38bdf5d-3131-4d09-bac3-fc11735adf16-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6\" (UID: \"f38bdf5d-3131-4d09-bac3-fc11735adf16\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:16.974899 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:16.974908    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f38bdf5d-3131-4d09-bac3-fc11735adf16-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6\" (UID: \"f38bdf5d-3131-4d09-bac3-fc11735adf16\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:16.975100 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:16.974932    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bjvs\" (UniqueName: \"kubernetes.io/projected/f38bdf5d-3131-4d09-bac3-fc11735adf16-kube-api-access-2bjvs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6\" (UID: \"f38bdf5d-3131-4d09-bac3-fc11735adf16\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:16.975145 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:16.975108    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f38bdf5d-3131-4d09-bac3-fc11735adf16-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6\" (UID: \"f38bdf5d-3131-4d09-bac3-fc11735adf16\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:16.975213 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:16.975198    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f38bdf5d-3131-4d09-bac3-fc11735adf16-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6\" (UID: \"f38bdf5d-3131-4d09-bac3-fc11735adf16\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:16.975278 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:16.975264    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f38bdf5d-3131-4d09-bac3-fc11735adf16-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6\" (UID: \"f38bdf5d-3131-4d09-bac3-fc11735adf16\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:17.018025 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:17.017963    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gw45w\""
Nov 04 01:43:17.018025 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:17.017971    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 04 01:43:17.018294 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:17.018056    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 04 01:43:17.019607 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:17.019586    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"]
Nov 04 01:43:17.075874 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:17.075841    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f38bdf5d-3131-4d09-bac3-fc11735adf16-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6\" (UID: \"f38bdf5d-3131-4d09-bac3-fc11735adf16\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:17.076102 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:17.075902    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f38bdf5d-3131-4d09-bac3-fc11735adf16-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6\" (UID: \"f38bdf5d-3131-4d09-bac3-fc11735adf16\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:17.076184 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:17.076120    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f38bdf5d-3131-4d09-bac3-fc11735adf16-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6\" (UID: \"f38bdf5d-3131-4d09-bac3-fc11735adf16\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:17.076184 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:17.076166    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f38bdf5d-3131-4d09-bac3-fc11735adf16-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6\" (UID: \"f38bdf5d-3131-4d09-bac3-fc11735adf16\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:17.076289 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:17.076195    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bjvs\" (UniqueName: \"kubernetes.io/projected/f38bdf5d-3131-4d09-bac3-fc11735adf16-kube-api-access-2bjvs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6\" (UID: \"f38bdf5d-3131-4d09-bac3-fc11735adf16\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:17.076289 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:17.076254    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f38bdf5d-3131-4d09-bac3-fc11735adf16-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6\" (UID: \"f38bdf5d-3131-4d09-bac3-fc11735adf16\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:17.076391 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:17.076367    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f38bdf5d-3131-4d09-bac3-fc11735adf16-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6\" (UID: \"f38bdf5d-3131-4d09-bac3-fc11735adf16\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:17.076527 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:17.076506    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f38bdf5d-3131-4d09-bac3-fc11735adf16-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6\" (UID: \"f38bdf5d-3131-4d09-bac3-fc11735adf16\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:17.076594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:17.076547    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f38bdf5d-3131-4d09-bac3-fc11735adf16-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6\" (UID: \"f38bdf5d-3131-4d09-bac3-fc11735adf16\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:17.078354 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:17.078335    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f38bdf5d-3131-4d09-bac3-fc11735adf16-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6\" (UID: \"f38bdf5d-3131-4d09-bac3-fc11735adf16\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:17.078550 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:17.078533    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f38bdf5d-3131-4d09-bac3-fc11735adf16-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6\" (UID: \"f38bdf5d-3131-4d09-bac3-fc11735adf16\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:17.137476 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:17.137421    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bjvs\" (UniqueName: \"kubernetes.io/projected/f38bdf5d-3131-4d09-bac3-fc11735adf16-kube-api-access-2bjvs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6\" (UID: \"f38bdf5d-3131-4d09-bac3-fc11735adf16\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:17.227408 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:17.227326    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:17.605892 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:17.605864    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"]
Nov 04 01:43:17.608381 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:43:17.608352    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf38bdf5d_3131_4d09_bac3_fc11735adf16.slice/crio-7c1a61f4b4a08c4382f08539e6afb000894f3e4ed99b521aa0aea27dc025dc20 WatchSource:0}: Error finding container 7c1a61f4b4a08c4382f08539e6afb000894f3e4ed99b521aa0aea27dc025dc20: Status 404 returned error can't find the container with id 7c1a61f4b4a08c4382f08539e6afb000894f3e4ed99b521aa0aea27dc025dc20
Nov 04 01:43:18.296775 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:18.296736    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6" event={"ID":"f38bdf5d-3131-4d09-bac3-fc11735adf16","Type":"ContainerStarted","Data":"7c1a61f4b4a08c4382f08539e6afb000894f3e4ed99b521aa0aea27dc025dc20"}
Nov 04 01:43:20.307954 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:20.307914    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6" event={"ID":"f38bdf5d-3131-4d09-bac3-fc11735adf16","Type":"ContainerStarted","Data":"a1d8e7848087c7959662f3c1591cd974a4858d7f185efe6c2dc02ccc96689c52"}
Nov 04 01:43:20.429868 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:20.429816    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6" podStartSLOduration=2.219682055 podStartE2EDuration="4.429802082s" totalImagesPullingTime="2.210120027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:43:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:43:20.428478264 +0000 UTC m=+485.549755791" watchObservedRunningTime="2026-11-04 01:43:20.429802082 +0000 UTC m=+485.551079607"
Nov 04 01:43:21.098969 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:21.098930    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:43:21.227489 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:21.227442    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:27.228130 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:27.228094    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:27.240432 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:27.240405    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:27.351922 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:27.351888    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6"
Nov 04 01:43:28.656492 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:28.653100    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:43:28.913851 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:28.913772    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"]
Nov 04 01:43:28.917353 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:28.917330    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:28.969686 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:28.969657    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Nov 04 01:43:28.976859 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:28.976816    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"]
Nov 04 01:43:28.983831 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:28.983800    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/255db273-54b3-4fa6-bdf1-e84cd275b05d-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xgzhk\" (UID: \"255db273-54b3-4fa6-bdf1-e84cd275b05d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:28.983831 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:28.983836    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/255db273-54b3-4fa6-bdf1-e84cd275b05d-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xgzhk\" (UID: \"255db273-54b3-4fa6-bdf1-e84cd275b05d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:28.984039 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:28.983859    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/255db273-54b3-4fa6-bdf1-e84cd275b05d-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xgzhk\" (UID: \"255db273-54b3-4fa6-bdf1-e84cd275b05d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:28.984079 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:28.984029    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4fk9\" (UniqueName: \"kubernetes.io/projected/255db273-54b3-4fa6-bdf1-e84cd275b05d-kube-api-access-g4fk9\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xgzhk\" (UID: \"255db273-54b3-4fa6-bdf1-e84cd275b05d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:28.984115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:28.984089    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/255db273-54b3-4fa6-bdf1-e84cd275b05d-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xgzhk\" (UID: \"255db273-54b3-4fa6-bdf1-e84cd275b05d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:28.984165 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:28.984149    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/255db273-54b3-4fa6-bdf1-e84cd275b05d-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xgzhk\" (UID: \"255db273-54b3-4fa6-bdf1-e84cd275b05d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:29.084853 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:29.084815    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/255db273-54b3-4fa6-bdf1-e84cd275b05d-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xgzhk\" (UID: \"255db273-54b3-4fa6-bdf1-e84cd275b05d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:29.085075 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:29.084896    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/255db273-54b3-4fa6-bdf1-e84cd275b05d-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xgzhk\" (UID: \"255db273-54b3-4fa6-bdf1-e84cd275b05d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:29.085075 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:29.084918    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/255db273-54b3-4fa6-bdf1-e84cd275b05d-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xgzhk\" (UID: \"255db273-54b3-4fa6-bdf1-e84cd275b05d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:29.085075 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:29.084944    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/255db273-54b3-4fa6-bdf1-e84cd275b05d-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xgzhk\" (UID: \"255db273-54b3-4fa6-bdf1-e84cd275b05d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:29.085075 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:29.084976    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4fk9\" (UniqueName: \"kubernetes.io/projected/255db273-54b3-4fa6-bdf1-e84cd275b05d-kube-api-access-g4fk9\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xgzhk\" (UID: \"255db273-54b3-4fa6-bdf1-e84cd275b05d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:29.085075 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:29.085018    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/255db273-54b3-4fa6-bdf1-e84cd275b05d-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xgzhk\" (UID: \"255db273-54b3-4fa6-bdf1-e84cd275b05d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:29.085492 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:29.085436    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/255db273-54b3-4fa6-bdf1-e84cd275b05d-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xgzhk\" (UID: \"255db273-54b3-4fa6-bdf1-e84cd275b05d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:29.085492 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:29.085485    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/255db273-54b3-4fa6-bdf1-e84cd275b05d-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xgzhk\" (UID: \"255db273-54b3-4fa6-bdf1-e84cd275b05d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:29.085801 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:29.085638    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/255db273-54b3-4fa6-bdf1-e84cd275b05d-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xgzhk\" (UID: \"255db273-54b3-4fa6-bdf1-e84cd275b05d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:29.087352 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:29.087327    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/255db273-54b3-4fa6-bdf1-e84cd275b05d-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xgzhk\" (UID: \"255db273-54b3-4fa6-bdf1-e84cd275b05d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:29.087441 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:29.087427    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/255db273-54b3-4fa6-bdf1-e84cd275b05d-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xgzhk\" (UID: \"255db273-54b3-4fa6-bdf1-e84cd275b05d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:29.146230 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:29.146203    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4fk9\" (UniqueName: \"kubernetes.io/projected/255db273-54b3-4fa6-bdf1-e84cd275b05d-kube-api-access-g4fk9\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xgzhk\" (UID: \"255db273-54b3-4fa6-bdf1-e84cd275b05d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:29.229890 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:29.229777    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:29.603728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:29.603702    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"]
Nov 04 01:43:29.606358 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:43:29.606330    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod255db273_54b3_4fa6_bdf1_e84cd275b05d.slice/crio-8e1ae28eb36510ad7de97d0984310e0ad75d8a8555d9833259cdda67eda46560 WatchSource:0}: Error finding container 8e1ae28eb36510ad7de97d0984310e0ad75d8a8555d9833259cdda67eda46560: Status 404 returned error can't find the container with id 8e1ae28eb36510ad7de97d0984310e0ad75d8a8555d9833259cdda67eda46560
Nov 04 01:43:30.356247 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:30.356217    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk" event={"ID":"255db273-54b3-4fa6-bdf1-e84cd275b05d","Type":"ContainerStarted","Data":"8e1ae28eb36510ad7de97d0984310e0ad75d8a8555d9833259cdda67eda46560"}
Nov 04 01:43:31.362122 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:31.362082    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk" event={"ID":"255db273-54b3-4fa6-bdf1-e84cd275b05d","Type":"ContainerStarted","Data":"7e6b69ecb72140b58c0ed032d6057859b963ba7f2ec90b4896e822af1f493a1b"}
Nov 04 01:43:31.478854 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:31.478757    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk" podStartSLOduration=2.768323086 podStartE2EDuration="3.478739182s" totalImagesPullingTime="710.416096ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:43:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:43:31.47834147 +0000 UTC m=+496.599618997" watchObservedRunningTime="2026-11-04 01:43:31.478739182 +0000 UTC m=+496.600016709"
Nov 04 01:43:32.230807 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.230763    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:32.748734 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.748699    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"]
Nov 04 01:43:32.752200 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.752181    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:32.822178 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.822146    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62f552a2-733d-419e-b9d2-6fa8a2246d6e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-z22gn\" (UID: \"62f552a2-733d-419e-b9d2-6fa8a2246d6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:32.822178 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.822183    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62f552a2-733d-419e-b9d2-6fa8a2246d6e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-z22gn\" (UID: \"62f552a2-733d-419e-b9d2-6fa8a2246d6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:32.822383 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.822308    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62f552a2-733d-419e-b9d2-6fa8a2246d6e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-z22gn\" (UID: \"62f552a2-733d-419e-b9d2-6fa8a2246d6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:32.822383 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.822335    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62f552a2-733d-419e-b9d2-6fa8a2246d6e-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-z22gn\" (UID: \"62f552a2-733d-419e-b9d2-6fa8a2246d6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:32.822383 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.822374    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mtw2\" (UniqueName: \"kubernetes.io/projected/62f552a2-733d-419e-b9d2-6fa8a2246d6e-kube-api-access-9mtw2\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-z22gn\" (UID: \"62f552a2-733d-419e-b9d2-6fa8a2246d6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:32.822552 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.822528    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62f552a2-733d-419e-b9d2-6fa8a2246d6e-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-z22gn\" (UID: \"62f552a2-733d-419e-b9d2-6fa8a2246d6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:32.854495 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.854425    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Nov 04 01:43:32.856213 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.856190    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"]
Nov 04 01:43:32.924056 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.924015    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62f552a2-733d-419e-b9d2-6fa8a2246d6e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-z22gn\" (UID: \"62f552a2-733d-419e-b9d2-6fa8a2246d6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:32.924056 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.924057    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62f552a2-733d-419e-b9d2-6fa8a2246d6e-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-z22gn\" (UID: \"62f552a2-733d-419e-b9d2-6fa8a2246d6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:32.924312 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.924082    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mtw2\" (UniqueName: \"kubernetes.io/projected/62f552a2-733d-419e-b9d2-6fa8a2246d6e-kube-api-access-9mtw2\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-z22gn\" (UID: \"62f552a2-733d-419e-b9d2-6fa8a2246d6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:32.924312 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.924107    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62f552a2-733d-419e-b9d2-6fa8a2246d6e-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-z22gn\" (UID: \"62f552a2-733d-419e-b9d2-6fa8a2246d6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:32.924312 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.924152    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62f552a2-733d-419e-b9d2-6fa8a2246d6e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-z22gn\" (UID: \"62f552a2-733d-419e-b9d2-6fa8a2246d6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:32.924312 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.924172    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62f552a2-733d-419e-b9d2-6fa8a2246d6e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-z22gn\" (UID: \"62f552a2-733d-419e-b9d2-6fa8a2246d6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:32.924553 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.924533    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62f552a2-733d-419e-b9d2-6fa8a2246d6e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-z22gn\" (UID: \"62f552a2-733d-419e-b9d2-6fa8a2246d6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:32.924618 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.924566    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62f552a2-733d-419e-b9d2-6fa8a2246d6e-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-z22gn\" (UID: \"62f552a2-733d-419e-b9d2-6fa8a2246d6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:32.924618 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.924594    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62f552a2-733d-419e-b9d2-6fa8a2246d6e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-z22gn\" (UID: \"62f552a2-733d-419e-b9d2-6fa8a2246d6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:32.926394 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.926377    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62f552a2-733d-419e-b9d2-6fa8a2246d6e-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-z22gn\" (UID: \"62f552a2-733d-419e-b9d2-6fa8a2246d6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:32.926724 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.926707    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62f552a2-733d-419e-b9d2-6fa8a2246d6e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-z22gn\" (UID: \"62f552a2-733d-419e-b9d2-6fa8a2246d6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:32.985194 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:32.985161    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mtw2\" (UniqueName: \"kubernetes.io/projected/62f552a2-733d-419e-b9d2-6fa8a2246d6e-kube-api-access-9mtw2\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-z22gn\" (UID: \"62f552a2-733d-419e-b9d2-6fa8a2246d6e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:33.064487 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:33.064376    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:33.439595 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:33.439413    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"]
Nov 04 01:43:33.442228 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:43:33.442194    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62f552a2_733d_419e_b9d2_6fa8a2246d6e.slice/crio-8afa89b75b5401e7e7dda9622aa39f321eb1946d5bf7299185e0c63d3538cdb4 WatchSource:0}: Error finding container 8afa89b75b5401e7e7dda9622aa39f321eb1946d5bf7299185e0c63d3538cdb4: Status 404 returned error can't find the container with id 8afa89b75b5401e7e7dda9622aa39f321eb1946d5bf7299185e0c63d3538cdb4
Nov 04 01:43:34.380070 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:34.379979    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn" event={"ID":"62f552a2-733d-419e-b9d2-6fa8a2246d6e","Type":"ContainerStarted","Data":"6afe66dc6534e8083cc5a555fc86a3550fdb41be20f105563e09ce45943e1482"}
Nov 04 01:43:34.380070 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:34.380029    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn" event={"ID":"62f552a2-733d-419e-b9d2-6fa8a2246d6e","Type":"ContainerStarted","Data":"8afa89b75b5401e7e7dda9622aa39f321eb1946d5bf7299185e0c63d3538cdb4"}
Nov 04 01:43:34.498350 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:34.498287    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn" podStartSLOduration=1.923339 podStartE2EDuration="2.498272052s" totalImagesPullingTime="574.933052ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:43:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:43:34.496616282 +0000 UTC m=+499.617893808" watchObservedRunningTime="2026-11-04 01:43:34.498272052 +0000 UTC m=+499.619549578"
Nov 04 01:43:35.065267 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:35.065233    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:38.162530 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:38.162494    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:43:39.230731 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:39.230688    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:39.243153 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:39.243127    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:39.414845 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:39.414817    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk"
Nov 04 01:43:41.991814 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:43:41.991776    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:43:43.064816 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:43.064784    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:43.077226 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:43.077200    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:43.434065 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:43.434035    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn"
Nov 04 01:43:49.544465 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.544414    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"]
Nov 04 01:43:49.550104 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.550082    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.589255 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.589220    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0ba288d-0e75-4e9e-888a-10c1847a4ef2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk\" (UID: \"b0ba288d-0e75-4e9e-888a-10c1847a4ef2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.589460 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.589263    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hn9d\" (UniqueName: \"kubernetes.io/projected/b0ba288d-0e75-4e9e-888a-10c1847a4ef2-kube-api-access-9hn9d\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk\" (UID: \"b0ba288d-0e75-4e9e-888a-10c1847a4ef2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.589460 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.589414    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0ba288d-0e75-4e9e-888a-10c1847a4ef2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk\" (UID: \"b0ba288d-0e75-4e9e-888a-10c1847a4ef2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.589558 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.589514    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0ba288d-0e75-4e9e-888a-10c1847a4ef2-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk\" (UID: \"b0ba288d-0e75-4e9e-888a-10c1847a4ef2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.589558 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.589531    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0ba288d-0e75-4e9e-888a-10c1847a4ef2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk\" (UID: \"b0ba288d-0e75-4e9e-888a-10c1847a4ef2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.589656 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.589556    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0ba288d-0e75-4e9e-888a-10c1847a4ef2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk\" (UID: \"b0ba288d-0e75-4e9e-888a-10c1847a4ef2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.604534 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.604508    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Nov 04 01:43:49.655799 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.655762    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"]
Nov 04 01:43:49.690540 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.690506    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0ba288d-0e75-4e9e-888a-10c1847a4ef2-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk\" (UID: \"b0ba288d-0e75-4e9e-888a-10c1847a4ef2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.690798 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.690770    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0ba288d-0e75-4e9e-888a-10c1847a4ef2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk\" (UID: \"b0ba288d-0e75-4e9e-888a-10c1847a4ef2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.692167 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.690812    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0ba288d-0e75-4e9e-888a-10c1847a4ef2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk\" (UID: \"b0ba288d-0e75-4e9e-888a-10c1847a4ef2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.692167 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.690899    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0ba288d-0e75-4e9e-888a-10c1847a4ef2-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk\" (UID: \"b0ba288d-0e75-4e9e-888a-10c1847a4ef2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.692167 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.690942    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0ba288d-0e75-4e9e-888a-10c1847a4ef2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk\" (UID: \"b0ba288d-0e75-4e9e-888a-10c1847a4ef2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.692167 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.690980    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hn9d\" (UniqueName: \"kubernetes.io/projected/b0ba288d-0e75-4e9e-888a-10c1847a4ef2-kube-api-access-9hn9d\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk\" (UID: \"b0ba288d-0e75-4e9e-888a-10c1847a4ef2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.692167 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.691049    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0ba288d-0e75-4e9e-888a-10c1847a4ef2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk\" (UID: \"b0ba288d-0e75-4e9e-888a-10c1847a4ef2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.692167 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.691321    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0ba288d-0e75-4e9e-888a-10c1847a4ef2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk\" (UID: \"b0ba288d-0e75-4e9e-888a-10c1847a4ef2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.692167 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.691672    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0ba288d-0e75-4e9e-888a-10c1847a4ef2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk\" (UID: \"b0ba288d-0e75-4e9e-888a-10c1847a4ef2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.693597 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.693575    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0ba288d-0e75-4e9e-888a-10c1847a4ef2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk\" (UID: \"b0ba288d-0e75-4e9e-888a-10c1847a4ef2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.694792 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.694771    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0ba288d-0e75-4e9e-888a-10c1847a4ef2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk\" (UID: \"b0ba288d-0e75-4e9e-888a-10c1847a4ef2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.752003 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.751965    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hn9d\" (UniqueName: \"kubernetes.io/projected/b0ba288d-0e75-4e9e-888a-10c1847a4ef2-kube-api-access-9hn9d\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk\" (UID: \"b0ba288d-0e75-4e9e-888a-10c1847a4ef2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:49.861840 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:49.861745    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:50.236514 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:50.236484    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"]
Nov 04 01:43:50.238552 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:43:50.238521    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0ba288d_0e75_4e9e_888a_10c1847a4ef2.slice/crio-d965ce1cb66fa1db8ddfcf30e7dcb54b77950a2ddff476b00b79bdd3621e9779 WatchSource:0}: Error finding container d965ce1cb66fa1db8ddfcf30e7dcb54b77950a2ddff476b00b79bdd3621e9779: Status 404 returned error can't find the container with id d965ce1cb66fa1db8ddfcf30e7dcb54b77950a2ddff476b00b79bdd3621e9779
Nov 04 01:43:50.458061 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:50.458023    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk" event={"ID":"b0ba288d-0e75-4e9e-888a-10c1847a4ef2","Type":"ContainerStarted","Data":"d965ce1cb66fa1db8ddfcf30e7dcb54b77950a2ddff476b00b79bdd3621e9779"}
Nov 04 01:43:51.464673 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:51.464638    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk" event={"ID":"b0ba288d-0e75-4e9e-888a-10c1847a4ef2","Type":"ContainerStarted","Data":"c74c31639246bc318a679954cbf579b7b2456af9b85cac5421b80526b3f19b6b"}
Nov 04 01:43:51.581240 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:51.581168    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk" podStartSLOduration=1.993809217 podStartE2EDuration="2.581149459s" totalImagesPullingTime="587.340242ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:43:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:43:51.580421919 +0000 UTC m=+516.701699446" watchObservedRunningTime="2026-11-04 01:43:51.581149459 +0000 UTC m=+516.702426985"
Nov 04 01:43:51.862344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:51.862260    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:55.267681 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:55.267649    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:43:59.862520 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:59.862480    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:43:59.874896 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:43:59.874871    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:44:00.516707 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:00.516678    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk"
Nov 04 01:44:04.336210 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.336171    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"]
Nov 04 01:44:04.339820 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.339802    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.385866 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.385826    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"]
Nov 04 01:44:04.389430 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.389406    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.415468 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.415423    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-plld6\""
Nov 04 01:44:04.430513 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.430478    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.430513 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.430517    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/829a880f-8e4d-40eb-ac51-309a55583ada-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.430700 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.430584    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.430700 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.430648    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.430700 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.430690    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.430800 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.430732    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.430800 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.430760    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.430873 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.430819    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.430873 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.430837    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.430873 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.430863    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.430983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.430883    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/829a880f-8e4d-40eb-ac51-309a55583ada-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.430983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.430966    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.431052 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.430983    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.431052 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.431014    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2j7g\" (UniqueName: \"kubernetes.io/projected/829a880f-8e4d-40eb-ac51-309a55583ada-kube-api-access-j2j7g\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.431129 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.431050    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.431169 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.431138    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzhp4\" (UniqueName: \"kubernetes.io/projected/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-kube-api-access-pzhp4\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.431169 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.431157    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.431242 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.431175    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/829a880f-8e4d-40eb-ac51-309a55583ada-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.469078 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.469039    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"]
Nov 04 01:44:04.518993 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.518958    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"]
Nov 04 01:44:04.531911 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.531854    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.532107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.531921    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.532107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.531945    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2j7g\" (UniqueName: \"kubernetes.io/projected/829a880f-8e4d-40eb-ac51-309a55583ada-kube-api-access-j2j7g\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.532107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.531963    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.532107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532017    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzhp4\" (UniqueName: \"kubernetes.io/projected/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-kube-api-access-pzhp4\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.532107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532037    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.532107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532051    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/829a880f-8e4d-40eb-ac51-309a55583ada-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.532473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532149    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.532473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532177    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/829a880f-8e4d-40eb-ac51-309a55583ada-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.532473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532197    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.532473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532224    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.532473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532255    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.532719 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532644    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.532719 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532698    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.532818 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532721    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.532818 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532733    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.532818 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532754    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.532818 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532789    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.532818 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532794    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.532818 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532812    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.533106 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532853    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.533106 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532884    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/829a880f-8e4d-40eb-ac51-309a55583ada-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.533106 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.532951    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.533106 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.533078    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.533106 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.533096    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/829a880f-8e4d-40eb-ac51-309a55583ada-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.533338 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.533190    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.533423 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.533406    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.533587 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.533566    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.535780 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.535520    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.535780 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.535735    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.535969 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.535806    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.536018 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.535961    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/829a880f-8e4d-40eb-ac51-309a55583ada-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.593824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.593748    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.593983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.593824    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzhp4\" (UniqueName: \"kubernetes.io/projected/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-kube-api-access-pzhp4\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbf55\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:04.594061 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.594045    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/829a880f-8e4d-40eb-ac51-309a55583ada-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.594161 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.594143    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2j7g\" (UniqueName: \"kubernetes.io/projected/829a880f-8e4d-40eb-ac51-309a55583ada-kube-api-access-j2j7g\") pod \"maas-default-gateway-openshift-default-778b7b5c47-hw8r9\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.653944 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.653906    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:04.702540 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:04.702500    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:05.081864 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:05.081831    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"]
Nov 04 01:44:05.083801 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:44:05.083771    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod829a880f_8e4d_40eb_ac51_309a55583ada.slice/crio-baf795926c3f739ceae7c99b50094b49f5464eb1afb800d41c4c35f5bf31b151 WatchSource:0}: Error finding container baf795926c3f739ceae7c99b50094b49f5464eb1afb800d41c4c35f5bf31b151: Status 404 returned error can't find the container with id baf795926c3f739ceae7c99b50094b49f5464eb1afb800d41c4c35f5bf31b151
Nov 04 01:44:05.085914 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:05.085880    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:44:05.085978 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:05.085955    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:44:05.086016 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:05.085985    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:44:05.134676 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:05.134649    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"]
Nov 04 01:44:05.137365 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:44:05.137325    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98c2fe44_752b_4134_8e4d_d4a3db4cdf5f.slice/crio-a9b2485a2c36222c093a0a702b3fa05ed9b537d893fa662c3bc00480348bb1c9 WatchSource:0}: Error finding container a9b2485a2c36222c093a0a702b3fa05ed9b537d893fa662c3bc00480348bb1c9: Status 404 returned error can't find the container with id a9b2485a2c36222c093a0a702b3fa05ed9b537d893fa662c3bc00480348bb1c9
Nov 04 01:44:05.139577 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:05.139545    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:44:05.139663 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:05.139619    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:44:05.139663 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:05.139650    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:44:05.533024 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:05.532981    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9" event={"ID":"829a880f-8e4d-40eb-ac51-309a55583ada","Type":"ContainerStarted","Data":"b79f3ca2d8e7da32b20225f9da43f9b78c117d5a05abe24979d5abf331a34591"}
Nov 04 01:44:05.533485 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:05.533052    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9" event={"ID":"829a880f-8e4d-40eb-ac51-309a55583ada","Type":"ContainerStarted","Data":"baf795926c3f739ceae7c99b50094b49f5464eb1afb800d41c4c35f5bf31b151"}
Nov 04 01:44:05.535412 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:05.535202    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55" event={"ID":"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f","Type":"ContainerStarted","Data":"b6cd75338d8a842847708038fc8e80d38a59e7261514694172919298514e132b"}
Nov 04 01:44:05.535412 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:05.535248    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55" event={"ID":"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f","Type":"ContainerStarted","Data":"a9b2485a2c36222c093a0a702b3fa05ed9b537d893fa662c3bc00480348bb1c9"}
Nov 04 01:44:05.655473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:05.654678    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:05.660473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:05.656476    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9" podStartSLOduration=1.656436454 podStartE2EDuration="1.656436454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:44:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:44:05.655017516 +0000 UTC m=+530.776295043" watchObservedRunningTime="2026-11-04 01:44:05.656436454 +0000 UTC m=+530.777713981"
Nov 04 01:44:05.703583 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:05.703539    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:05.789538 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:05.788802    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55" podStartSLOduration=1.788782713 podStartE2EDuration="1.788782713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:44:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:44:05.786048441 +0000 UTC m=+530.907326000" watchObservedRunningTime="2026-11-04 01:44:05.788782713 +0000 UTC m=+530.910060239"
Nov 04 01:44:06.663261 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:06.663225    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-hw8r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Nov 04 01:44:06.663886 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:06.663844    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9" podUID="829a880f-8e4d-40eb-ac51-309a55583ada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Nov 04 01:44:06.704592 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:06.704557    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xbf55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Nov 04 01:44:06.704756 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:06.704613    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55" podUID="98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Nov 04 01:44:07.658473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:07.657892    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-hw8r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Nov 04 01:44:07.658473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:07.657956    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9" podUID="829a880f-8e4d-40eb-ac51-309a55583ada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Nov 04 01:44:07.703693 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:07.703665    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xbf55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Nov 04 01:44:07.703758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:07.703716    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55" podUID="98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Nov 04 01:44:08.655472 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:08.654851    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-hw8r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Nov 04 01:44:08.655472 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:08.654916    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9" podUID="829a880f-8e4d-40eb-ac51-309a55583ada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Nov 04 01:44:08.703649 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:08.703611    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xbf55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Nov 04 01:44:08.704025 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:08.703670    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55" podUID="98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Nov 04 01:44:09.676475 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:09.665946    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-hw8r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Nov 04 01:44:09.676475 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:09.666016    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9" podUID="829a880f-8e4d-40eb-ac51-309a55583ada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Nov 04 01:44:09.702956 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:09.702919    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xbf55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Nov 04 01:44:09.703137 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:09.702974    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55" podUID="98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Nov 04 01:44:10.658475 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:10.654915    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-hw8r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Nov 04 01:44:10.658475 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:10.654982    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9" podUID="829a880f-8e4d-40eb-ac51-309a55583ada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Nov 04 01:44:10.702999 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:10.702953    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xbf55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Nov 04 01:44:10.703177 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:10.703018    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55" podUID="98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Nov 04 01:44:11.654904 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:11.654870    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-hw8r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Nov 04 01:44:11.655106 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:11.654925    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9" podUID="829a880f-8e4d-40eb-ac51-309a55583ada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Nov 04 01:44:11.703462 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:11.703423    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xbf55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Nov 04 01:44:11.703845 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:11.703490    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55" podUID="98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Nov 04 01:44:11.992303 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:44:11.992210    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:44:12.654859 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:12.654816    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-hw8r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Nov 04 01:44:12.655050 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:12.654886    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9" podUID="829a880f-8e4d-40eb-ac51-309a55583ada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Nov 04 01:44:12.703439 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:12.703398    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xbf55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Nov 04 01:44:12.703629 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:12.703486    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55" podUID="98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Nov 04 01:44:13.658294 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:13.658264    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:13.658602 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:13.658581    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:13.659273 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:13.659255    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:44:13.707598 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:13.707565    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:13.707966 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:13.707853    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:13.708742 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:44:13.708722    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:44:41.991574 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:44:41.991536    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:45:00.176248 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:00.176211    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx"]
Nov 04 01:45:00.179723 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:00.179705    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx"
Nov 04 01:45:00.204798 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:00.204767    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf4qb\" (UniqueName: \"kubernetes.io/projected/f07ff75f-eb59-4688-a6d5-ce6be5efe9c9-kube-api-access-cf4qb\") pod \"maas-api-key-cleanup-29895945-d8ppx\" (UID: \"f07ff75f-eb59-4688-a6d5-ce6be5efe9c9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx"
Nov 04 01:45:00.204925 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:00.204833    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f07ff75f-eb59-4688-a6d5-ce6be5efe9c9-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895945-d8ppx\" (UID: \"f07ff75f-eb59-4688-a6d5-ce6be5efe9c9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx"
Nov 04 01:45:00.234910 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:00.234884    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:45:00.237519 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:00.237488    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx"]
Nov 04 01:45:00.281680 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:00.281652    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:45:00.281848 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:00.281747    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bj8b6\""
Nov 04 01:45:00.305941 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:00.305909    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf4qb\" (UniqueName: \"kubernetes.io/projected/f07ff75f-eb59-4688-a6d5-ce6be5efe9c9-kube-api-access-cf4qb\") pod \"maas-api-key-cleanup-29895945-d8ppx\" (UID: \"f07ff75f-eb59-4688-a6d5-ce6be5efe9c9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx"
Nov 04 01:45:00.306094 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:00.305968    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f07ff75f-eb59-4688-a6d5-ce6be5efe9c9-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895945-d8ppx\" (UID: \"f07ff75f-eb59-4688-a6d5-ce6be5efe9c9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx"
Nov 04 01:45:00.306595 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:00.306578    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f07ff75f-eb59-4688-a6d5-ce6be5efe9c9-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895945-d8ppx\" (UID: \"f07ff75f-eb59-4688-a6d5-ce6be5efe9c9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx"
Nov 04 01:45:00.366838 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:00.366808    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf4qb\" (UniqueName: \"kubernetes.io/projected/f07ff75f-eb59-4688-a6d5-ce6be5efe9c9-kube-api-access-cf4qb\") pod \"maas-api-key-cleanup-29895945-d8ppx\" (UID: \"f07ff75f-eb59-4688-a6d5-ce6be5efe9c9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx"
Nov 04 01:45:00.491842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:00.491752    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx"
Nov 04 01:45:00.863330 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:00.863305    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx"]
Nov 04 01:45:00.865549 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:45:00.865514    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf07ff75f_eb59_4688_a6d5_ce6be5efe9c9.slice/crio-44691433453bf76be5382479c5096df715746c7c6d249763f4c072c58b555e44 WatchSource:0}: Error finding container 44691433453bf76be5382479c5096df715746c7c6d249763f4c072c58b555e44: Status 404 returned error can't find the container with id 44691433453bf76be5382479c5096df715746c7c6d249763f4c072c58b555e44
Nov 04 01:45:01.853045 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:01.853011    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx" event={"ID":"f07ff75f-eb59-4688-a6d5-ce6be5efe9c9","Type":"ContainerStarted","Data":"44691433453bf76be5382479c5096df715746c7c6d249763f4c072c58b555e44"}
Nov 04 01:45:05.875023 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:05.874988    6116 generic.go:356] "Generic (PLEG): container finished" podID="f07ff75f-eb59-4688-a6d5-ce6be5efe9c9" containerID="85233c8062e143e676ca6a7754a5b8c0cacc4b59d8b329070ec1f077ae528b24" exitCode=60
Nov 04 01:45:05.875491 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:05.875081    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx" event={"ID":"f07ff75f-eb59-4688-a6d5-ce6be5efe9c9","Type":"ContainerDied","Data":"85233c8062e143e676ca6a7754a5b8c0cacc4b59d8b329070ec1f077ae528b24"}
Nov 04 01:45:05.875491 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:05.875359    6116 scope.go:122] "RemoveContainer" containerID="85233c8062e143e676ca6a7754a5b8c0cacc4b59d8b329070ec1f077ae528b24"
Nov 04 01:45:06.882616 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:06.882507    6116 generic.go:356] "Generic (PLEG): container finished" podID="f07ff75f-eb59-4688-a6d5-ce6be5efe9c9" containerID="8e10eb5adb3600421b60cfe7078d14bf0ca9eba327190a0678e3dc67fd7510fe" exitCode=60
Nov 04 01:45:06.882616 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:06.882565    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx" event={"ID":"f07ff75f-eb59-4688-a6d5-ce6be5efe9c9","Type":"ContainerDied","Data":"8e10eb5adb3600421b60cfe7078d14bf0ca9eba327190a0678e3dc67fd7510fe"}
Nov 04 01:45:06.883136 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:06.882638    6116 scope.go:122] "RemoveContainer" containerID="85233c8062e143e676ca6a7754a5b8c0cacc4b59d8b329070ec1f077ae528b24"
Nov 04 01:45:06.883136 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:06.882858    6116 scope.go:122] "RemoveContainer" containerID="8e10eb5adb3600421b60cfe7078d14bf0ca9eba327190a0678e3dc67fd7510fe"
Nov 04 01:45:06.883136 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:45:06.883111    6116 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895945-d8ppx_odh-ai-gateway-infra(f07ff75f-eb59-4688-a6d5-ce6be5efe9c9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx" podUID="f07ff75f-eb59-4688-a6d5-ce6be5efe9c9"
Nov 04 01:45:07.257535 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.257500    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"]
Nov 04 01:45:07.262154 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.262129    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.316376 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.316348    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-82706f-openshift-default-dockercfg-v862d\""
Nov 04 01:45:07.322321 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.322289    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"]
Nov 04 01:45:07.377645 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.377606    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.377805 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.377665    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.377805 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.377734    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.377903 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.377825    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.377903 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.377877    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt5lv\" (UniqueName: \"kubernetes.io/projected/30488fc4-bfaf-414c-9dcd-6e250398b8dc-kube-api-access-bt5lv\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.377903 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.377898    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.377991 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.377979    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.378055 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.378034    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.378115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.378104    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.479585 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.479548    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.479585 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.479594    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.479837 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.479610    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.479837 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.479644    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.479837 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.479684    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt5lv\" (UniqueName: \"kubernetes.io/projected/30488fc4-bfaf-414c-9dcd-6e250398b8dc-kube-api-access-bt5lv\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.479837 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.479711    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.479837 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.479767    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.479837 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.479804    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.480074 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.479916    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.480074 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.480046    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.480261 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.480238    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.480261 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.480245    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.480482 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.480359    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.480943 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.480917    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.482634 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.482612    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.482867 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.482846    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.541109 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.541043    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt5lv\" (UniqueName: \"kubernetes.io/projected/30488fc4-bfaf-414c-9dcd-6e250398b8dc-kube-api-access-bt5lv\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.543824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.543794    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.574360 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.574297    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:07.889615 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.889535    6116 scope.go:122] "RemoveContainer" containerID="8e10eb5adb3600421b60cfe7078d14bf0ca9eba327190a0678e3dc67fd7510fe"
Nov 04 01:45:07.889966 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:45:07.889757    6116 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895945-d8ppx_odh-ai-gateway-infra(f07ff75f-eb59-4688-a6d5-ce6be5efe9c9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx" podUID="f07ff75f-eb59-4688-a6d5-ce6be5efe9c9"
Nov 04 01:45:07.905758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.905726    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"]
Nov 04 01:45:07.910615 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.910592    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:07.962936 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.962905    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-f4d202-openshift-default-dockercfg-6vjrk\""
Nov 04 01:45:07.970180 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.970141    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"]
Nov 04 01:45:07.984805 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.984774    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-data\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:07.985018 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.984815    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-envoy\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:07.985018 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.984891    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-credential-socket\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:07.985018 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.984923    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-workload-certs\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:07.985018 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.984952    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-workload-socket\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:07.985251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.985074    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-token\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:07.985251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.985201    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-podinfo\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:07.985251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.985238    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istiod-ca-cert\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:07.985410 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:07.985254    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s8dd\" (UniqueName: \"kubernetes.io/projected/006f9c41-8228-4a5e-9a98-3afb19a40cbc-kube-api-access-5s8dd\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.010527 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.010496    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"]
Nov 04 01:45:08.011846 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:45:08.011818    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30488fc4_bfaf_414c_9dcd_6e250398b8dc.slice/crio-a23193b18b8e624d0e66b0dbd33b4a97b0af9b7492539ebbf7c6a5eaba82552a WatchSource:0}: Error finding container a23193b18b8e624d0e66b0dbd33b4a97b0af9b7492539ebbf7c6a5eaba82552a: Status 404 returned error can't find the container with id a23193b18b8e624d0e66b0dbd33b4a97b0af9b7492539ebbf7c6a5eaba82552a
Nov 04 01:45:08.014184 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.014152    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:45:08.014275 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.014223    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:45:08.014275 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.014250    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:45:08.086132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.086093    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-podinfo\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.086295 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.086140    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istiod-ca-cert\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.086295 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.086155    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s8dd\" (UniqueName: \"kubernetes.io/projected/006f9c41-8228-4a5e-9a98-3afb19a40cbc-kube-api-access-5s8dd\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.086295 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.086212    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-data\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.086295 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.086235    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-envoy\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.086295 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.086256    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-credential-socket\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.086295 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.086275    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-workload-certs\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.086622 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.086314    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-workload-socket\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.086622 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.086343    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-token\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.086841 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.086818    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-workload-certs\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.086911 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.086849    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-workload-socket\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.086911 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.086875    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-credential-socket\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.086986 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.086917    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istiod-ca-cert\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.086986 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.086979    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-data\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.088790 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.088757    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-envoy\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.089122 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.089103    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-podinfo\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.147360 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.147288    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-token\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.147531 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.147358    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s8dd\" (UniqueName: \"kubernetes.io/projected/006f9c41-8228-4a5e-9a98-3afb19a40cbc-kube-api-access-5s8dd\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.224262 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.224223    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:08.666347 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.666319    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"]
Nov 04 01:45:08.668258 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:45:08.668233    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod006f9c41_8228_4a5e_9a98_3afb19a40cbc.slice/crio-f2e5a10f549cafae69dcf592edd79aa8db6af2e7d2c35df9bca73a272980fbbb WatchSource:0}: Error finding container f2e5a10f549cafae69dcf592edd79aa8db6af2e7d2c35df9bca73a272980fbbb: Status 404 returned error can't find the container with id f2e5a10f549cafae69dcf592edd79aa8db6af2e7d2c35df9bca73a272980fbbb
Nov 04 01:45:08.670854 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.670808    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:45:08.670945 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.670905    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:45:08.670984 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.670950    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:45:08.896744 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.896694    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2" event={"ID":"006f9c41-8228-4a5e-9a98-3afb19a40cbc","Type":"ContainerStarted","Data":"35ad9202b4c9a79236b9f41841dee7b5229eface16f6ba95a809554dce2b1fb7"}
Nov 04 01:45:08.897211 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.896762    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2" event={"ID":"006f9c41-8228-4a5e-9a98-3afb19a40cbc","Type":"ContainerStarted","Data":"f2e5a10f549cafae69dcf592edd79aa8db6af2e7d2c35df9bca73a272980fbbb"}
Nov 04 01:45:08.898473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.898404    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k" event={"ID":"30488fc4-bfaf-414c-9dcd-6e250398b8dc","Type":"ContainerStarted","Data":"b2abc865032b5f7b32978de46dfcf3c5cc8fcf9664faaf52d2c89b58bdd23f2e"}
Nov 04 01:45:08.898567 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:08.898499    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k" event={"ID":"30488fc4-bfaf-414c-9dcd-6e250398b8dc","Type":"ContainerStarted","Data":"a23193b18b8e624d0e66b0dbd33b4a97b0af9b7492539ebbf7c6a5eaba82552a"}
Nov 04 01:45:09.020959 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:09.020862    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2" podStartSLOduration=2.020847637 podStartE2EDuration="2.020847637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:45:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:45:09.016072069 +0000 UTC m=+594.137349593" watchObservedRunningTime="2026-11-04 01:45:09.020847637 +0000 UTC m=+594.142125160"
Nov 04 01:45:09.143185 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:09.143117    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k" podStartSLOduration=2.143097797 podStartE2EDuration="2.143097797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:45:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:45:09.140767513 +0000 UTC m=+594.262045039" watchObservedRunningTime="2026-11-04 01:45:09.143097797 +0000 UTC m=+594.264375325"
Nov 04 01:45:09.224813 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:09.224771    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:09.229912 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:09.229883    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:09.575466 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:09.575426    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:09.580623 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:09.580598    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:09.904895 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:09.904864    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:09.905297 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:09.904926    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:09.905787 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:09.905769    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:09.905867 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:09.905822    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:11.991831 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:45:11.991791    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:45:15.104191 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:15.104162    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/prometheus/0.log"
Nov 04 01:45:15.104788 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:15.104294    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/prometheus/0.log"
Nov 04 01:45:15.160370 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:15.160335    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6lfbv_192e2fec-fefb-4bc5-9516-ed7464963a10/ovn-acl-logging/0.log"
Nov 04 01:45:15.161057 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:15.161035    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6lfbv_192e2fec-fefb-4bc5-9516-ed7464963a10/ovn-acl-logging/0.log"
Nov 04 01:45:16.024540 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:16.024502    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x"]
Nov 04 01:45:16.028567 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:16.028542    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x"
Nov 04 01:45:16.132768 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:16.132728    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-f4d202-dockercfg-vdngc\""
Nov 04 01:45:16.133528 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:16.132794    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-f4d202\""
Nov 04 01:45:16.135080 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:16.135057    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x"]
Nov 04 01:45:16.171811 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:16.171772    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dn2f\" (UniqueName: \"kubernetes.io/projected/af11b6f0-9314-4cb7-ac0d-76ecf4bb665d-kube-api-access-4dn2f\") pod \"payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x\" (UID: \"af11b6f0-9314-4cb7-ac0d-76ecf4bb665d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x"
Nov 04 01:45:16.171987 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:16.171875    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af11b6f0-9314-4cb7-ac0d-76ecf4bb665d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x\" (UID: \"af11b6f0-9314-4cb7-ac0d-76ecf4bb665d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x"
Nov 04 01:45:16.273623 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:16.273585    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dn2f\" (UniqueName: \"kubernetes.io/projected/af11b6f0-9314-4cb7-ac0d-76ecf4bb665d-kube-api-access-4dn2f\") pod \"payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x\" (UID: \"af11b6f0-9314-4cb7-ac0d-76ecf4bb665d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x"
Nov 04 01:45:16.273623 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:16.273627    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af11b6f0-9314-4cb7-ac0d-76ecf4bb665d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x\" (UID: \"af11b6f0-9314-4cb7-ac0d-76ecf4bb665d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x"
Nov 04 01:45:16.274289 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:16.274269    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af11b6f0-9314-4cb7-ac0d-76ecf4bb665d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x\" (UID: \"af11b6f0-9314-4cb7-ac0d-76ecf4bb665d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x"
Nov 04 01:45:16.334356 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:16.334282    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dn2f\" (UniqueName: \"kubernetes.io/projected/af11b6f0-9314-4cb7-ac0d-76ecf4bb665d-kube-api-access-4dn2f\") pod \"payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x\" (UID: \"af11b6f0-9314-4cb7-ac0d-76ecf4bb665d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x"
Nov 04 01:45:16.341207 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:16.341180    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x"
Nov 04 01:45:16.715274 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:16.715251    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x"]
Nov 04 01:45:16.717414 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:45:16.717386    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf11b6f0_9314_4cb7_ac0d_76ecf4bb665d.slice/crio-bcc89b9af7b6acd688bbe7c945abd7e2a9bbc96f3c98da5fad58dc4571478387 WatchSource:0}: Error finding container bcc89b9af7b6acd688bbe7c945abd7e2a9bbc96f3c98da5fad58dc4571478387: Status 404 returned error can't find the container with id bcc89b9af7b6acd688bbe7c945abd7e2a9bbc96f3c98da5fad58dc4571478387
Nov 04 01:45:16.946281 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:16.946242    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x" event={"ID":"af11b6f0-9314-4cb7-ac0d-76ecf4bb665d","Type":"ContainerStarted","Data":"5d1fd3747cb3e01a6bd088e5fd85d601802f79ab944f019bf0ce375afcd152a8"}
Nov 04 01:45:16.946493 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:16.946308    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x" event={"ID":"af11b6f0-9314-4cb7-ac0d-76ecf4bb665d","Type":"ContainerStarted","Data":"bcc89b9af7b6acd688bbe7c945abd7e2a9bbc96f3c98da5fad58dc4571478387"}
Nov 04 01:45:16.946493 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:16.946332    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x"
Nov 04 01:45:17.061889 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:17.061790    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x" podStartSLOduration=2.061775789 podStartE2EDuration="2.061775789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:45:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:45:17.060343774 +0000 UTC m=+602.181621300" watchObservedRunningTime="2026-11-04 01:45:17.061775789 +0000 UTC m=+602.183053314"
Nov 04 01:45:19.021140 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:19.021102    6116 scope.go:122] "RemoveContainer" containerID="8e10eb5adb3600421b60cfe7078d14bf0ca9eba327190a0678e3dc67fd7510fe"
Nov 04 01:45:19.963405 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:19.963373    6116 generic.go:356] "Generic (PLEG): container finished" podID="f07ff75f-eb59-4688-a6d5-ce6be5efe9c9" containerID="0f47a99be74f54ede1921aea8a37e1a562c79e98020820e1280005c6446bb3a6" exitCode=60
Nov 04 01:45:19.963589 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:19.963436    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx" event={"ID":"f07ff75f-eb59-4688-a6d5-ce6be5efe9c9","Type":"ContainerDied","Data":"0f47a99be74f54ede1921aea8a37e1a562c79e98020820e1280005c6446bb3a6"}
Nov 04 01:45:19.963589 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:19.963531    6116 scope.go:122] "RemoveContainer" containerID="8e10eb5adb3600421b60cfe7078d14bf0ca9eba327190a0678e3dc67fd7510fe"
Nov 04 01:45:19.963853 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:19.963838    6116 scope.go:122] "RemoveContainer" containerID="0f47a99be74f54ede1921aea8a37e1a562c79e98020820e1280005c6446bb3a6"
Nov 04 01:45:19.964113 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:45:19.964089    6116 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29895945-d8ppx_odh-ai-gateway-infra(f07ff75f-eb59-4688-a6d5-ce6be5efe9c9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx" podUID="f07ff75f-eb59-4688-a6d5-ce6be5efe9c9"
Nov 04 01:45:21.012276 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.012238    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw"]
Nov 04 01:45:21.017665 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.017618    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw"
Nov 04 01:45:21.049492 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.049442    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw"]
Nov 04 01:45:21.072994 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.072966    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-82706f-dockercfg-qxqw7\""
Nov 04 01:45:21.075832 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.075795    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-82706f\""
Nov 04 01:45:21.078294 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.078271    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42"]
Nov 04 01:45:21.081893 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.081875    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42"
Nov 04 01:45:21.092737 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.092713    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx"]
Nov 04 01:45:21.132862 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.132826    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbmfd\" (UniqueName: \"kubernetes.io/projected/e3bcfc78-b4c1-4448-b848-a6400e8b6912-kube-api-access-gbmfd\") pod \"payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw\" (UID: \"e3bcfc78-b4c1-4448-b848-a6400e8b6912\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw"
Nov 04 01:45:21.133064 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.133039    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3bcfc78-b4c1-4448-b848-a6400e8b6912-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw\" (UID: \"e3bcfc78-b4c1-4448-b848-a6400e8b6912\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw"
Nov 04 01:45:21.139511 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.139484    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42"]
Nov 04 01:45:21.235146 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.235114    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe49df1c-2c17-440d-8e83-3b6ec337e909-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42\" (UID: \"fe49df1c-2c17-440d-8e83-3b6ec337e909\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42"
Nov 04 01:45:21.235298 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.235245    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3bcfc78-b4c1-4448-b848-a6400e8b6912-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw\" (UID: \"e3bcfc78-b4c1-4448-b848-a6400e8b6912\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw"
Nov 04 01:45:21.235385 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.235368    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szcrp\" (UniqueName: \"kubernetes.io/projected/fe49df1c-2c17-440d-8e83-3b6ec337e909-kube-api-access-szcrp\") pod \"payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42\" (UID: \"fe49df1c-2c17-440d-8e83-3b6ec337e909\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42"
Nov 04 01:45:21.235460 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.235412    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbmfd\" (UniqueName: \"kubernetes.io/projected/e3bcfc78-b4c1-4448-b848-a6400e8b6912-kube-api-access-gbmfd\") pod \"payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw\" (UID: \"e3bcfc78-b4c1-4448-b848-a6400e8b6912\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw"
Nov 04 01:45:21.236293 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.236266    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3bcfc78-b4c1-4448-b848-a6400e8b6912-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw\" (UID: \"e3bcfc78-b4c1-4448-b848-a6400e8b6912\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw"
Nov 04 01:45:21.257377 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.257346    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx"
Nov 04 01:45:21.353540 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.351699    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f07ff75f-eb59-4688-a6d5-ce6be5efe9c9-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f07ff75f-eb59-4688-a6d5-ce6be5efe9c9-maas-api-service-ca\") pod \"f07ff75f-eb59-4688-a6d5-ce6be5efe9c9\" (UID: \"f07ff75f-eb59-4688-a6d5-ce6be5efe9c9\") "
Nov 04 01:45:21.353540 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.351773    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f07ff75f-eb59-4688-a6d5-ce6be5efe9c9-kube-api-access-cf4qb\" (UniqueName: \"kubernetes.io/projected/f07ff75f-eb59-4688-a6d5-ce6be5efe9c9-kube-api-access-cf4qb\") pod \"f07ff75f-eb59-4688-a6d5-ce6be5efe9c9\" (UID: \"f07ff75f-eb59-4688-a6d5-ce6be5efe9c9\") "
Nov 04 01:45:21.353540 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.352001    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szcrp\" (UniqueName: \"kubernetes.io/projected/fe49df1c-2c17-440d-8e83-3b6ec337e909-kube-api-access-szcrp\") pod \"payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42\" (UID: \"fe49df1c-2c17-440d-8e83-3b6ec337e909\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42"
Nov 04 01:45:21.353540 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.352058    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbmfd\" (UniqueName: \"kubernetes.io/projected/e3bcfc78-b4c1-4448-b848-a6400e8b6912-kube-api-access-gbmfd\") pod \"payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw\" (UID: \"e3bcfc78-b4c1-4448-b848-a6400e8b6912\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw"
Nov 04 01:45:21.353540 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.352111    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe49df1c-2c17-440d-8e83-3b6ec337e909-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42\" (UID: \"fe49df1c-2c17-440d-8e83-3b6ec337e909\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42"
Nov 04 01:45:21.353540 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.352643    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f07ff75f-eb59-4688-a6d5-ce6be5efe9c9-maas-api-service-ca" pod "f07ff75f-eb59-4688-a6d5-ce6be5efe9c9" (UID: "f07ff75f-eb59-4688-a6d5-ce6be5efe9c9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:45:21.353540 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.352921    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe49df1c-2c17-440d-8e83-3b6ec337e909-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42\" (UID: \"fe49df1c-2c17-440d-8e83-3b6ec337e909\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42"
Nov 04 01:45:21.364981 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.364926    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f07ff75f-eb59-4688-a6d5-ce6be5efe9c9-kube-api-access-cf4qb" pod "f07ff75f-eb59-4688-a6d5-ce6be5efe9c9" (UID: "f07ff75f-eb59-4688-a6d5-ce6be5efe9c9"). InnerVolumeSpecName "kube-api-access-cf4qb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:45:21.415810 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.415776    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szcrp\" (UniqueName: \"kubernetes.io/projected/fe49df1c-2c17-440d-8e83-3b6ec337e909-kube-api-access-szcrp\") pod \"payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42\" (UID: \"fe49df1c-2c17-440d-8e83-3b6ec337e909\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42"
Nov 04 01:45:21.453576 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.453540    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cf4qb\" (UniqueName: \"kubernetes.io/projected/f07ff75f-eb59-4688-a6d5-ce6be5efe9c9-kube-api-access-cf4qb\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:45:21.453576 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.453574    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f07ff75f-eb59-4688-a6d5-ce6be5efe9c9-maas-api-service-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:45:21.649167 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.649073    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw"
Nov 04 01:45:21.696334 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.696300    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42"
Nov 04 01:45:21.977605 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.977576    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx" event={"ID":"f07ff75f-eb59-4688-a6d5-ce6be5efe9c9","Type":"ContainerDied","Data":"44691433453bf76be5382479c5096df715746c7c6d249763f4c072c58b555e44"}
Nov 04 01:45:21.977777 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.977608    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx"
Nov 04 01:45:21.977777 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:21.977669    6116 scope.go:122] "RemoveContainer" containerID="0f47a99be74f54ede1921aea8a37e1a562c79e98020820e1280005c6446bb3a6"
Nov 04 01:45:22.023083 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:22.023054    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw"]
Nov 04 01:45:22.024526 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:45:22.024497    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3bcfc78_b4c1_4448_b848_a6400e8b6912.slice/crio-57af1f65f2a65f9708157c4ddee375b9d4d656bacd9190220817da091dcd5263 WatchSource:0}: Error finding container 57af1f65f2a65f9708157c4ddee375b9d4d656bacd9190220817da091dcd5263: Status 404 returned error can't find the container with id 57af1f65f2a65f9708157c4ddee375b9d4d656bacd9190220817da091dcd5263
Nov 04 01:45:22.080667 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:22.080642    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42"]
Nov 04 01:45:22.082118 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:45:22.082090    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe49df1c_2c17_440d_8e83_3b6ec337e909.slice/crio-3166368cd7a37495c28e7299ce97de0e9092c79cae6679a95caaeb458b903470 WatchSource:0}: Error finding container 3166368cd7a37495c28e7299ce97de0e9092c79cae6679a95caaeb458b903470: Status 404 returned error can't find the container with id 3166368cd7a37495c28e7299ce97de0e9092c79cae6679a95caaeb458b903470
Nov 04 01:45:22.144153 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:22.144123    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx"]
Nov 04 01:45:22.149309 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:22.149277    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895945-d8ppx"]
Nov 04 01:45:22.984517 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:22.984479    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw" event={"ID":"e3bcfc78-b4c1-4448-b848-a6400e8b6912","Type":"ContainerStarted","Data":"f283355c22bd884bfd699ffc09e08512ade7622f90b997a5fbb2f00f21ca2038"}
Nov 04 01:45:22.984729 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:22.984549    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw" event={"ID":"e3bcfc78-b4c1-4448-b848-a6400e8b6912","Type":"ContainerStarted","Data":"57af1f65f2a65f9708157c4ddee375b9d4d656bacd9190220817da091dcd5263"}
Nov 04 01:45:22.984729 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:22.984591    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw"
Nov 04 01:45:22.986327 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:22.986295    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42" event={"ID":"fe49df1c-2c17-440d-8e83-3b6ec337e909","Type":"ContainerStarted","Data":"e3a70a3836348e8b6bdb5d91e139a2486501599b54b66c82ee02164ddf5c2375"}
Nov 04 01:45:22.986507 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:22.986349    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42" event={"ID":"fe49df1c-2c17-440d-8e83-3b6ec337e909","Type":"ContainerStarted","Data":"3166368cd7a37495c28e7299ce97de0e9092c79cae6679a95caaeb458b903470"}
Nov 04 01:45:22.986507 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:22.986406    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42"
Nov 04 01:45:23.027414 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.027379    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f07ff75f-eb59-4688-a6d5-ce6be5efe9c9" path="/var/lib/kubelet/pods/f07ff75f-eb59-4688-a6d5-ce6be5efe9c9/volumes"
Nov 04 01:45:23.104403 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.104326    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw" podStartSLOduration=3.104311527 podStartE2EDuration="3.104311527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:45:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:45:23.102920305 +0000 UTC m=+608.224197828" watchObservedRunningTime="2026-11-04 01:45:23.104311527 +0000 UTC m=+608.225589053"
Nov 04 01:45:23.218002 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.217941    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42" podStartSLOduration=2.217923149 podStartE2EDuration="2.217923149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:45:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:45:23.217432551 +0000 UTC m=+608.338710076" watchObservedRunningTime="2026-11-04 01:45:23.217923149 +0000 UTC m=+608.339200677"
Nov 04 01:45:23.454708 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.454669    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4"]
Nov 04 01:45:23.455135 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.455122    6116 cpu_manager.go:414] "containerMap: removing container" podUID="f07ff75f-eb59-4688-a6d5-ce6be5efe9c9" containerName="cleanup"
Nov 04 01:45:23.455179 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.455137    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f07ff75f-eb59-4688-a6d5-ce6be5efe9c9" containerName="cleanup"
Nov 04 01:45:23.455179 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.455158    6116 cpu_manager.go:414] "containerMap: removing container" podUID="f07ff75f-eb59-4688-a6d5-ce6be5efe9c9" containerName="cleanup"
Nov 04 01:45:23.455179 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.455163    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f07ff75f-eb59-4688-a6d5-ce6be5efe9c9" containerName="cleanup"
Nov 04 01:45:23.455179 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.455178    6116 cpu_manager.go:414] "containerMap: removing container" podUID="f07ff75f-eb59-4688-a6d5-ce6be5efe9c9" containerName="cleanup"
Nov 04 01:45:23.455293 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.455183    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f07ff75f-eb59-4688-a6d5-ce6be5efe9c9" containerName="cleanup"
Nov 04 01:45:23.455293 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.455241    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="f07ff75f-eb59-4688-a6d5-ce6be5efe9c9" containerName="cleanup"
Nov 04 01:45:23.455293 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.455250    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="f07ff75f-eb59-4688-a6d5-ce6be5efe9c9" containerName="cleanup"
Nov 04 01:45:23.455293 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.455258    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="f07ff75f-eb59-4688-a6d5-ce6be5efe9c9" containerName="cleanup"
Nov 04 01:45:23.458383 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.458361    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4"
Nov 04 01:45:23.515987 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.515952    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4"]
Nov 04 01:45:23.516199 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.516175    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-fa0149\""
Nov 04 01:45:23.516419 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.516398    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-fa0149-dockercfg-drjhs\""
Nov 04 01:45:23.578097 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.578059    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjwjd\" (UniqueName: \"kubernetes.io/projected/79e5a497-975f-4f5f-b76f-20213e9f202d-kube-api-access-hjwjd\") pod \"payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4\" (UID: \"79e5a497-975f-4f5f-b76f-20213e9f202d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4"
Nov 04 01:45:23.578463 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.578419    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79e5a497-975f-4f5f-b76f-20213e9f202d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4\" (UID: \"79e5a497-975f-4f5f-b76f-20213e9f202d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4"
Nov 04 01:45:23.608394 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.608360    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v"]
Nov 04 01:45:23.612328 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.612304    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v"
Nov 04 01:45:23.680016 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.679982    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79e5a497-975f-4f5f-b76f-20213e9f202d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4\" (UID: \"79e5a497-975f-4f5f-b76f-20213e9f202d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4"
Nov 04 01:45:23.680178 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.680045    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjwjd\" (UniqueName: \"kubernetes.io/projected/79e5a497-975f-4f5f-b76f-20213e9f202d-kube-api-access-hjwjd\") pod \"payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4\" (UID: \"79e5a497-975f-4f5f-b76f-20213e9f202d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4"
Nov 04 01:45:23.680712 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.680689    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79e5a497-975f-4f5f-b76f-20213e9f202d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4\" (UID: \"79e5a497-975f-4f5f-b76f-20213e9f202d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4"
Nov 04 01:45:23.701014 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.700980    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v"]
Nov 04 01:45:23.759061 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.758995    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjwjd\" (UniqueName: \"kubernetes.io/projected/79e5a497-975f-4f5f-b76f-20213e9f202d-kube-api-access-hjwjd\") pod \"payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4\" (UID: \"79e5a497-975f-4f5f-b76f-20213e9f202d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4"
Nov 04 01:45:23.772000 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.771974    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4"
Nov 04 01:45:23.780709 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.780683    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5xh7\" (UniqueName: \"kubernetes.io/projected/7ba76e9b-c899-45e6-85a6-fcaffea854a4-kube-api-access-v5xh7\") pod \"payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v\" (UID: \"7ba76e9b-c899-45e6-85a6-fcaffea854a4\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v"
Nov 04 01:45:23.780816 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.780734    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ba76e9b-c899-45e6-85a6-fcaffea854a4-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v\" (UID: \"7ba76e9b-c899-45e6-85a6-fcaffea854a4\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v"
Nov 04 01:45:23.882310 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.882275    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ba76e9b-c899-45e6-85a6-fcaffea854a4-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v\" (UID: \"7ba76e9b-c899-45e6-85a6-fcaffea854a4\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v"
Nov 04 01:45:23.882495 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.882422    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5xh7\" (UniqueName: \"kubernetes.io/projected/7ba76e9b-c899-45e6-85a6-fcaffea854a4-kube-api-access-v5xh7\") pod \"payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v\" (UID: \"7ba76e9b-c899-45e6-85a6-fcaffea854a4\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v"
Nov 04 01:45:23.883004 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.882985    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ba76e9b-c899-45e6-85a6-fcaffea854a4-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v\" (UID: \"7ba76e9b-c899-45e6-85a6-fcaffea854a4\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v"
Nov 04 01:45:23.943298 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:23.943268    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5xh7\" (UniqueName: \"kubernetes.io/projected/7ba76e9b-c899-45e6-85a6-fcaffea854a4-kube-api-access-v5xh7\") pod \"payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v\" (UID: \"7ba76e9b-c899-45e6-85a6-fcaffea854a4\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v"
Nov 04 01:45:24.145279 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:24.145244    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4"]
Nov 04 01:45:24.146784 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:45:24.146758    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79e5a497_975f_4f5f_b76f_20213e9f202d.slice/crio-1891fb8f522718ec14df1e4dbda366515dd2a7d3a73fc07f96030beff2fd2650 WatchSource:0}: Error finding container 1891fb8f522718ec14df1e4dbda366515dd2a7d3a73fc07f96030beff2fd2650: Status 404 returned error can't find the container with id 1891fb8f522718ec14df1e4dbda366515dd2a7d3a73fc07f96030beff2fd2650
Nov 04 01:45:24.225094 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:24.225064    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v"
Nov 04 01:45:24.602055 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:24.602029    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v"]
Nov 04 01:45:24.604169 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:45:24.604144    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ba76e9b_c899_45e6_85a6_fcaffea854a4.slice/crio-20b7346f3865b96648035dfbb39bb738a1be8b96f52a93182409812f2dfd36c2 WatchSource:0}: Error finding container 20b7346f3865b96648035dfbb39bb738a1be8b96f52a93182409812f2dfd36c2: Status 404 returned error can't find the container with id 20b7346f3865b96648035dfbb39bb738a1be8b96f52a93182409812f2dfd36c2
Nov 04 01:45:25.001925 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:25.001886    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v" event={"ID":"7ba76e9b-c899-45e6-85a6-fcaffea854a4","Type":"ContainerStarted","Data":"2919ca11159f88abdb64f533d45f5f52047689d56bf9b3b25cc05a77e579cf2b"}
Nov 04 01:45:25.002127 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:25.001951    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v" event={"ID":"7ba76e9b-c899-45e6-85a6-fcaffea854a4","Type":"ContainerStarted","Data":"20b7346f3865b96648035dfbb39bb738a1be8b96f52a93182409812f2dfd36c2"}
Nov 04 01:45:25.002127 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:25.002038    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v"
Nov 04 01:45:25.003531 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:25.003505    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4" event={"ID":"79e5a497-975f-4f5f-b76f-20213e9f202d","Type":"ContainerStarted","Data":"b0b59c99f0b7ed51bb08fabd420a651e252b300d0c3c3b6dcf73479ac8740a37"}
Nov 04 01:45:25.003655 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:25.003544    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4" event={"ID":"79e5a497-975f-4f5f-b76f-20213e9f202d","Type":"ContainerStarted","Data":"1891fb8f522718ec14df1e4dbda366515dd2a7d3a73fc07f96030beff2fd2650"}
Nov 04 01:45:25.003655 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:25.003582    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4"
Nov 04 01:45:25.118949 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:25.118896    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v" podStartSLOduration=2.118882448 podStartE2EDuration="2.118882448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:45:25.116597592 +0000 UTC m=+610.237875116" watchObservedRunningTime="2026-11-04 01:45:25.118882448 +0000 UTC m=+610.240159973"
Nov 04 01:45:25.237727 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:25.237660    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4" podStartSLOduration=2.237645934 podStartE2EDuration="2.237645934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:45:25.236479912 +0000 UTC m=+610.357757441" watchObservedRunningTime="2026-11-04 01:45:25.237645934 +0000 UTC m=+610.358923460"
Nov 04 01:45:26.080294 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.080252    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt"]
Nov 04 01:45:26.083848 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.083825    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt"
Nov 04 01:45:26.105759 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.105727    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f051c9f2-e56c-4ced-b88d-b85747f3254f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt\" (UID: \"f051c9f2-e56c-4ced-b88d-b85747f3254f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt"
Nov 04 01:45:26.106093 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.106064    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qdwj\" (UniqueName: \"kubernetes.io/projected/f051c9f2-e56c-4ced-b88d-b85747f3254f-kube-api-access-9qdwj\") pod \"payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt\" (UID: \"f051c9f2-e56c-4ced-b88d-b85747f3254f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt"
Nov 04 01:45:26.141969 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.141935    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt"]
Nov 04 01:45:26.188318 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.188279    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-7af38e\""
Nov 04 01:45:26.188517 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.188381    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-7af38e-dockercfg-d6qtp\""
Nov 04 01:45:26.207157 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.207123    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qdwj\" (UniqueName: \"kubernetes.io/projected/f051c9f2-e56c-4ced-b88d-b85747f3254f-kube-api-access-9qdwj\") pod \"payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt\" (UID: \"f051c9f2-e56c-4ced-b88d-b85747f3254f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt"
Nov 04 01:45:26.207326 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.207205    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f051c9f2-e56c-4ced-b88d-b85747f3254f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt\" (UID: \"f051c9f2-e56c-4ced-b88d-b85747f3254f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt"
Nov 04 01:45:26.207915 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.207894    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f051c9f2-e56c-4ced-b88d-b85747f3254f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt\" (UID: \"f051c9f2-e56c-4ced-b88d-b85747f3254f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt"
Nov 04 01:45:26.240142 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.240106    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh"]
Nov 04 01:45:26.244045 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.244028    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh"
Nov 04 01:45:26.269496 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.269441    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qdwj\" (UniqueName: \"kubernetes.io/projected/f051c9f2-e56c-4ced-b88d-b85747f3254f-kube-api-access-9qdwj\") pod \"payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt\" (UID: \"f051c9f2-e56c-4ced-b88d-b85747f3254f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt"
Nov 04 01:45:26.275696 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.275667    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh"]
Nov 04 01:45:26.308460 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.308413    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d8t2\" (UniqueName: \"kubernetes.io/projected/aaa5bb34-fdba-497a-b58b-9ca549fa3d18-kube-api-access-6d8t2\") pod \"payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh\" (UID: \"aaa5bb34-fdba-497a-b58b-9ca549fa3d18\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh"
Nov 04 01:45:26.308629 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.308549    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aaa5bb34-fdba-497a-b58b-9ca549fa3d18-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh\" (UID: \"aaa5bb34-fdba-497a-b58b-9ca549fa3d18\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh"
Nov 04 01:45:26.396564 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.396487    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt"
Nov 04 01:45:26.409536 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.409510    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d8t2\" (UniqueName: \"kubernetes.io/projected/aaa5bb34-fdba-497a-b58b-9ca549fa3d18-kube-api-access-6d8t2\") pod \"payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh\" (UID: \"aaa5bb34-fdba-497a-b58b-9ca549fa3d18\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh"
Nov 04 01:45:26.409654 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.409576    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aaa5bb34-fdba-497a-b58b-9ca549fa3d18-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh\" (UID: \"aaa5bb34-fdba-497a-b58b-9ca549fa3d18\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh"
Nov 04 01:45:26.410399 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.410375    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aaa5bb34-fdba-497a-b58b-9ca549fa3d18-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh\" (UID: \"aaa5bb34-fdba-497a-b58b-9ca549fa3d18\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh"
Nov 04 01:45:26.470255 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.470227    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d8t2\" (UniqueName: \"kubernetes.io/projected/aaa5bb34-fdba-497a-b58b-9ca549fa3d18-kube-api-access-6d8t2\") pod \"payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh\" (UID: \"aaa5bb34-fdba-497a-b58b-9ca549fa3d18\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh"
Nov 04 01:45:26.557678 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.557639    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh"
Nov 04 01:45:26.774476 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.774424    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt"]
Nov 04 01:45:26.774991 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:45:26.774961    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf051c9f2_e56c_4ced_b88d_b85747f3254f.slice/crio-4416f89c48782e1d197c37e4ad0c5e24863964fb58bed0c0fa269c15729e276d WatchSource:0}: Error finding container 4416f89c48782e1d197c37e4ad0c5e24863964fb58bed0c0fa269c15729e276d: Status 404 returned error can't find the container with id 4416f89c48782e1d197c37e4ad0c5e24863964fb58bed0c0fa269c15729e276d
Nov 04 01:45:26.928852 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:26.928828    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh"]
Nov 04 01:45:26.931413 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:45:26.931387    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaa5bb34_fdba_497a_b58b_9ca549fa3d18.slice/crio-2741cf04ff3a780a57116204463431946e13bed3f6b3f0f0bdb31d53aedcd31a WatchSource:0}: Error finding container 2741cf04ff3a780a57116204463431946e13bed3f6b3f0f0bdb31d53aedcd31a: Status 404 returned error can't find the container with id 2741cf04ff3a780a57116204463431946e13bed3f6b3f0f0bdb31d53aedcd31a
Nov 04 01:45:27.015487 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:27.015425    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt" event={"ID":"f051c9f2-e56c-4ced-b88d-b85747f3254f","Type":"ContainerStarted","Data":"d044c4c1dfc6471107413392659fa3c409bdf6f836e56b5c1c26568b7f0f6f35"}
Nov 04 01:45:27.015667 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:27.015513    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt"
Nov 04 01:45:27.015667 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:27.015537    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt" event={"ID":"f051c9f2-e56c-4ced-b88d-b85747f3254f","Type":"ContainerStarted","Data":"4416f89c48782e1d197c37e4ad0c5e24863964fb58bed0c0fa269c15729e276d"}
Nov 04 01:45:27.016997 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:27.016976    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh" event={"ID":"aaa5bb34-fdba-497a-b58b-9ca549fa3d18","Type":"ContainerStarted","Data":"f7bce7de817c496175624931d6f5883e8d21d0290825947f97952edc8ab38da4"}
Nov 04 01:45:27.017091 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:27.017011    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh" event={"ID":"aaa5bb34-fdba-497a-b58b-9ca549fa3d18","Type":"ContainerStarted","Data":"2741cf04ff3a780a57116204463431946e13bed3f6b3f0f0bdb31d53aedcd31a"}
Nov 04 01:45:27.017091 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:27.017030    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh"
Nov 04 01:45:27.130396 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:27.130263    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt" podStartSLOduration=1.13024553 podStartE2EDuration="1.13024553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:45:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:45:27.129488104 +0000 UTC m=+612.250765626" watchObservedRunningTime="2026-11-04 01:45:27.13024553 +0000 UTC m=+612.251523150"
Nov 04 01:45:27.249385 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:27.248677    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh" podStartSLOduration=1.248658607 podStartE2EDuration="1.248658607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:45:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:45:27.247937677 +0000 UTC m=+612.369215204" watchObservedRunningTime="2026-11-04 01:45:27.248658607 +0000 UTC m=+612.369936133"
Nov 04 01:45:27.671156 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:27.671119    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:45:27.973527 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:27.973435    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x"
Nov 04 01:45:33.377053 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:33.377018    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl"]
Nov 04 01:45:33.381861 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:33.381843    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl"
Nov 04 01:45:33.487461 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:33.485422    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-795045\""
Nov 04 01:45:33.489232 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:33.489186    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl"]
Nov 04 01:45:33.490320 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:33.490285    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-795045-dockercfg-l4plm\""
Nov 04 01:45:33.506896 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:33.506863    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02295850-87a3-45a0-aa2e-1f42985f6c79-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl\" (UID: \"02295850-87a3-45a0-aa2e-1f42985f6c79\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl"
Nov 04 01:45:33.507036 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:33.507001    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6dh5\" (UniqueName: \"kubernetes.io/projected/02295850-87a3-45a0-aa2e-1f42985f6c79-kube-api-access-t6dh5\") pod \"payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl\" (UID: \"02295850-87a3-45a0-aa2e-1f42985f6c79\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl"
Nov 04 01:45:33.608001 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:33.607947    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02295850-87a3-45a0-aa2e-1f42985f6c79-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl\" (UID: \"02295850-87a3-45a0-aa2e-1f42985f6c79\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl"
Nov 04 01:45:33.608176 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:33.608063    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6dh5\" (UniqueName: \"kubernetes.io/projected/02295850-87a3-45a0-aa2e-1f42985f6c79-kube-api-access-t6dh5\") pod \"payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl\" (UID: \"02295850-87a3-45a0-aa2e-1f42985f6c79\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl"
Nov 04 01:45:33.609136 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:33.609104    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02295850-87a3-45a0-aa2e-1f42985f6c79-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl\" (UID: \"02295850-87a3-45a0-aa2e-1f42985f6c79\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl"
Nov 04 01:45:33.669089 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:33.669050    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6dh5\" (UniqueName: \"kubernetes.io/projected/02295850-87a3-45a0-aa2e-1f42985f6c79-kube-api-access-t6dh5\") pod \"payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl\" (UID: \"02295850-87a3-45a0-aa2e-1f42985f6c79\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl"
Nov 04 01:45:33.695040 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:33.695012    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl"
Nov 04 01:45:33.997716 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:33.997687    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42"
Nov 04 01:45:33.997879 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:33.997763    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw"
Nov 04 01:45:34.071234 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:34.071203    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl"]
Nov 04 01:45:34.073264 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:45:34.073233    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02295850_87a3_45a0_aa2e_1f42985f6c79.slice/crio-31c43cc38d960626643f402e6d01fd404483ec482b5011f0a5bb2e83447fb5af WatchSource:0}: Error finding container 31c43cc38d960626643f402e6d01fd404483ec482b5011f0a5bb2e83447fb5af: Status 404 returned error can't find the container with id 31c43cc38d960626643f402e6d01fd404483ec482b5011f0a5bb2e83447fb5af
Nov 04 01:45:35.062117 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:35.062084    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl" event={"ID":"02295850-87a3-45a0-aa2e-1f42985f6c79","Type":"ContainerStarted","Data":"9cb6c726bc5a361540d13d9ee6e1c13b1ef3a0c0d45bd692b16d083b8d0eb85b"}
Nov 04 01:45:35.062552 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:35.062138    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl" event={"ID":"02295850-87a3-45a0-aa2e-1f42985f6c79","Type":"ContainerStarted","Data":"31c43cc38d960626643f402e6d01fd404483ec482b5011f0a5bb2e83447fb5af"}
Nov 04 01:45:35.062552 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:35.062244    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl"
Nov 04 01:45:35.176864 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:35.176805    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl" podStartSLOduration=2.176787613 podStartE2EDuration="2.176787613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:45:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:45:35.176556955 +0000 UTC m=+620.297834482" watchObservedRunningTime="2026-11-04 01:45:35.176787613 +0000 UTC m=+620.298065138"
Nov 04 01:45:36.012529 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:36.012495    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v"
Nov 04 01:45:36.012924 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:36.012902    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4"
Nov 04 01:45:36.452960 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:36.452917    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:45:37.074713 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:37.074679    6116 generic.go:356] "Generic (PLEG): container finished" podID="30488fc4-bfaf-414c-9dcd-6e250398b8dc" containerID="b2abc865032b5f7b32978de46dfcf3c5cc8fcf9664faaf52d2c89b58bdd23f2e" exitCode=0
Nov 04 01:45:37.074920 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:37.074759    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k" event={"ID":"30488fc4-bfaf-414c-9dcd-6e250398b8dc","Type":"ContainerDied","Data":"b2abc865032b5f7b32978de46dfcf3c5cc8fcf9664faaf52d2c89b58bdd23f2e"}
Nov 04 01:45:37.075203 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:37.075185    6116 scope.go:122] "RemoveContainer" containerID="b2abc865032b5f7b32978de46dfcf3c5cc8fcf9664faaf52d2c89b58bdd23f2e"
Nov 04 01:45:37.076498 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:37.076443    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:45:37.076558 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:37.076540    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:45:37.076592 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:37.076568    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:45:38.025619 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:38.025592    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh"
Nov 04 01:45:38.026044 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:38.025650    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt"
Nov 04 01:45:38.083284 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:38.083253    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k" event={"ID":"30488fc4-bfaf-414c-9dcd-6e250398b8dc","Type":"ContainerStarted","Data":"c0665c99a090db1e0d631877d792d6995d18434c3c1e8ecc01fe0e128e817861"}
Nov 04 01:45:38.575098 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:38.575064    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:38.576325 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:38.576303    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 04 01:45:38.576434 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:38.576347    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k" podUID="30488fc4-bfaf-414c-9dcd-6e250398b8dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 04 01:45:39.575429 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:39.575388    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 04 01:45:39.575874 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:39.575481    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k" podUID="30488fc4-bfaf-414c-9dcd-6e250398b8dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 04 01:45:39.905008 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:39.904979    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:40.575567 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:40.575527    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 04 01:45:40.575984 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:40.575601    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k" podUID="30488fc4-bfaf-414c-9dcd-6e250398b8dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 04 01:45:41.574751 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:41.574713    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 04 01:45:41.574937 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:41.574800    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k" podUID="30488fc4-bfaf-414c-9dcd-6e250398b8dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 04 01:45:41.995275 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:45:41.995237    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:45:42.574937 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:42.574902    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 04 01:45:42.575210 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:42.575187    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k" podUID="30488fc4-bfaf-414c-9dcd-6e250398b8dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 04 01:45:43.579319 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:43.579286    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:43.580386 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:43.580366    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:45:44.637129 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:44.637095    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:45:46.074236 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:46.074205    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl"
Nov 04 01:45:54.172110 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:54.172077    6116 generic.go:356] "Generic (PLEG): container finished" podID="006f9c41-8228-4a5e-9a98-3afb19a40cbc" containerID="35ad9202b4c9a79236b9f41841dee7b5229eface16f6ba95a809554dce2b1fb7" exitCode=0
Nov 04 01:45:54.172675 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:54.172133    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2" event={"ID":"006f9c41-8228-4a5e-9a98-3afb19a40cbc","Type":"ContainerDied","Data":"35ad9202b4c9a79236b9f41841dee7b5229eface16f6ba95a809554dce2b1fb7"}
Nov 04 01:45:54.172675 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:54.172610    6116 scope.go:122] "RemoveContainer" containerID="35ad9202b4c9a79236b9f41841dee7b5229eface16f6ba95a809554dce2b1fb7"
Nov 04 01:45:54.173862 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:54.173830    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:45:54.173952 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:54.173904    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:45:54.173952 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:54.173936    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:45:54.905358 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:54.905323    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:55.180050 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:55.179958    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2" event={"ID":"006f9c41-8228-4a5e-9a98-3afb19a40cbc","Type":"ContainerStarted","Data":"2e2e9bbe2e93b55254877f94981f6bbe1a1778044783326ef2d90ca751813bb4"}
Nov 04 01:45:55.224991 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:55.224963    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:45:55.226278 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:55.226252    6116 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Nov 04 01:45:55.226408 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:55.226310    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2" podUID="006f9c41-8228-4a5e-9a98-3afb19a40cbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Nov 04 01:45:56.225216 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:56.225180    6116 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Nov 04 01:45:56.225681 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:56.225233    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2" podUID="006f9c41-8228-4a5e-9a98-3afb19a40cbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Nov 04 01:45:57.224825 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:57.224789    6116 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Nov 04 01:45:57.225012 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:57.224841    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2" podUID="006f9c41-8228-4a5e-9a98-3afb19a40cbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Nov 04 01:45:58.225527 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:58.225497    6116 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Nov 04 01:45:58.225907 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:58.225549    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2" podUID="006f9c41-8228-4a5e-9a98-3afb19a40cbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Nov 04 01:45:59.224931 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:59.224894    6116 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Nov 04 01:45:59.225110 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:45:59.224947    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2" podUID="006f9c41-8228-4a5e-9a98-3afb19a40cbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Nov 04 01:46:00.229272 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:00.229246    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:46:00.229645 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:00.229630    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:46:00.230344 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:00.230330    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:46:00.558781 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:00.558700    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:46:02.645195 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.645160    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"]
Nov 04 01:46:02.648366 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.648349    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.744848 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.744807    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7w4t\" (UniqueName: \"kubernetes.io/projected/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-kube-api-access-k7w4t\") pod \"simulator-w2-82706f-kserve-646c6c569d-phzfp\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") " pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.744848 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.744851    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-tls-certs\") pod \"simulator-w2-82706f-kserve-646c6c569d-phzfp\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") " pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.745085 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.744937    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-model-cache\") pod \"simulator-w2-82706f-kserve-646c6c569d-phzfp\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") " pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.745085 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.745029    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-home\") pod \"simulator-w2-82706f-kserve-646c6c569d-phzfp\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") " pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.745154 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.745084    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-tmp-dir\") pod \"simulator-w2-82706f-kserve-646c6c569d-phzfp\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") " pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.745154 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.745113    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-dshm\") pod \"simulator-w2-82706f-kserve-646c6c569d-phzfp\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") " pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.750314 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.750283    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-82706f\"/\"default-dockercfg-r4744\""
Nov 04 01:46:02.750314 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.750304    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-82706f\"/\"simulator-w2-82706f-kserve-self-signed-certs\""
Nov 04 01:46:02.750534 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.750338    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-82706f\"/\"openshift-service-ca.crt\""
Nov 04 01:46:02.750534 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.750421    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-82706f\"/\"kube-root-ca.crt\""
Nov 04 01:46:02.752579 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.752560    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"]
Nov 04 01:46:02.846199 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.846160    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-tmp-dir\") pod \"simulator-w2-82706f-kserve-646c6c569d-phzfp\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") " pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.846199 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.846210    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-dshm\") pod \"simulator-w2-82706f-kserve-646c6c569d-phzfp\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") " pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.846482 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.846271    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7w4t\" (UniqueName: \"kubernetes.io/projected/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-kube-api-access-k7w4t\") pod \"simulator-w2-82706f-kserve-646c6c569d-phzfp\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") " pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.846482 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.846289    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-tls-certs\") pod \"simulator-w2-82706f-kserve-646c6c569d-phzfp\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") " pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.846482 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.846330    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-model-cache\") pod \"simulator-w2-82706f-kserve-646c6c569d-phzfp\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") " pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.846482 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.846368    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-home\") pod \"simulator-w2-82706f-kserve-646c6c569d-phzfp\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") " pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.846690 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.846617    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-tmp-dir\") pod \"simulator-w2-82706f-kserve-646c6c569d-phzfp\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") " pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.846690 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.846668    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-model-cache\") pod \"simulator-w2-82706f-kserve-646c6c569d-phzfp\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") " pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.846784 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.846728    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-home\") pod \"simulator-w2-82706f-kserve-646c6c569d-phzfp\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") " pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.848547 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.848516    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-dshm\") pod \"simulator-w2-82706f-kserve-646c6c569d-phzfp\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") " pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.848880 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.848864    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-tls-certs\") pod \"simulator-w2-82706f-kserve-646c6c569d-phzfp\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") " pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.909071 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.909034    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7w4t\" (UniqueName: \"kubernetes.io/projected/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-kube-api-access-k7w4t\") pod \"simulator-w2-82706f-kserve-646c6c569d-phzfp\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") " pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:02.959502 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:02.959443    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:03.332943 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:03.332906    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"]
Nov 04 01:46:03.334353 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:46:03.334327    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3dbce40_4ed1_4237_aa6c_8018e49f5e9e.slice/crio-05e027641cfc15332f94971b454ea1d2da2f127b9e1e5908b1f7b885d1189dd2 WatchSource:0}: Error finding container 05e027641cfc15332f94971b454ea1d2da2f127b9e1e5908b1f7b885d1189dd2: Status 404 returned error can't find the container with id 05e027641cfc15332f94971b454ea1d2da2f127b9e1e5908b1f7b885d1189dd2
Nov 04 01:46:04.232524 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:04.232486    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp" event={"ID":"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e","Type":"ContainerStarted","Data":"034829695f889e6da2d69849614067be3a55a6ffb27e92af4eba045e98a5d32b"}
Nov 04 01:46:04.232524 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:04.232531    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp" event={"ID":"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e","Type":"ContainerStarted","Data":"05e027641cfc15332f94971b454ea1d2da2f127b9e1e5908b1f7b885d1189dd2"}
Nov 04 01:46:04.350212 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:04.350151    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp" podStartSLOduration=2.350136752 podStartE2EDuration="2.350136752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:46:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:46:04.349886935 +0000 UTC m=+649.471164460" watchObservedRunningTime="2026-11-04 01:46:04.350136752 +0000 UTC m=+649.471414279"
Nov 04 01:46:04.960295 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:04.960260    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:11.991535 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:46:11.991498    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:46:12.959566 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:12.959526    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:12.972023 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:12.971996    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:13.295278 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:13.295200    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:46:15.743727 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:15.743691    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:46:27.802924 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:27.802884    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"]
Nov 04 01:46:27.805696 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:27.805664    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:27.858323 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:27.858296    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-fa0149\"/\"kube-root-ca.crt\""
Nov 04 01:46:27.875607 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:27.875574    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-fa0149\"/\"openshift-service-ca.crt\""
Nov 04 01:46:27.907164 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:27.907128    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-home\") pod \"simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") " pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:27.907327 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:27.907181    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-model-cache\") pod \"simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") " pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:27.907327 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:27.907239    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-tmp-dir\") pod \"simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") " pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:27.907327 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:27.907314    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdlx9\" (UniqueName: \"kubernetes.io/projected/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-kube-api-access-qdlx9\") pod \"simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") " pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:27.907480 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:27.907334    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-dshm\") pod \"simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") " pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:27.907480 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:27.907352    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-tls-certs\") pod \"simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") " pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:27.908186 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:27.908156    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-fa0149\"/\"simulator-w3-fa0149-kserve-self-signed-certs\""
Nov 04 01:46:27.908301 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:27.908233    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-fa0149\"/\"default-dockercfg-9f9nf\""
Nov 04 01:46:27.910354 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:27.910336    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"]
Nov 04 01:46:28.008333 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:28.008301    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-model-cache\") pod \"simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") " pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:28.008333 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:28.008337    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-tmp-dir\") pod \"simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") " pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:28.008597 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:28.008405    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdlx9\" (UniqueName: \"kubernetes.io/projected/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-kube-api-access-qdlx9\") pod \"simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") " pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:28.008597 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:28.008425    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-dshm\") pod \"simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") " pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:28.008597 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:28.008468    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-tls-certs\") pod \"simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") " pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:28.008597 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:28.008535    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-home\") pod \"simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") " pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:28.008861 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:28.008786    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-model-cache\") pod \"simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") " pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:28.008861 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:28.008813    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-tmp-dir\") pod \"simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") " pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:28.008972 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:28.008925    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-home\") pod \"simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") " pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:28.010859 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:28.010833    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-dshm\") pod \"simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") " pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:28.011095 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:28.011074    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-tls-certs\") pod \"simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") " pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:28.073083 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:28.073012    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdlx9\" (UniqueName: \"kubernetes.io/projected/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-kube-api-access-qdlx9\") pod \"simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") " pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:28.117672 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:28.117638    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:28.495243 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:28.495220    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"]
Nov 04 01:46:28.496943 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:46:28.496920    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1c8cce3_72a2_4d03_8680_42e8cd439f3b.slice/crio-17b82445b918af9f5b672b6d73e41f4d0eeac3efa3cafc2ab5ec9798730cd592 WatchSource:0}: Error finding container 17b82445b918af9f5b672b6d73e41f4d0eeac3efa3cafc2ab5ec9798730cd592: Status 404 returned error can't find the container with id 17b82445b918af9f5b672b6d73e41f4d0eeac3efa3cafc2ab5ec9798730cd592
Nov 04 01:46:29.372676 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:29.372634    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf" event={"ID":"e1c8cce3-72a2-4d03-8680-42e8cd439f3b","Type":"ContainerStarted","Data":"28a3fe6fc50610d5a47e2ebedfb5742454c171e43c57f1af4fdff3186b228ddb"}
Nov 04 01:46:29.372676 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:29.372678    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf" event={"ID":"e1c8cce3-72a2-4d03-8680-42e8cd439f3b","Type":"ContainerStarted","Data":"17b82445b918af9f5b672b6d73e41f4d0eeac3efa3cafc2ab5ec9798730cd592"}
Nov 04 01:46:29.489722 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:29.489671    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf" podStartSLOduration=2.489655567 podStartE2EDuration="2.489655567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:46:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:46:29.48874367 +0000 UTC m=+674.610021196" watchObservedRunningTime="2026-11-04 01:46:29.489655567 +0000 UTC m=+674.610933092"
Nov 04 01:46:30.118528 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:30.118485    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:33.742094 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:33.742061    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:46:38.118621 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:38.118586    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:38.130863 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:38.130838    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:38.435490 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:38.435437    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:46:41.990877 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:46:41.990843    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:46:42.634830 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.634798    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"]
Nov 04 01:46:42.637587 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.637565    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.691000 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.690969    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-f4d202\"/\"openshift-service-ca.crt\""
Nov 04 01:46:42.740585 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.740543    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-f4d202\"/\"simulator-w4-f4d202-kserve-self-signed-certs\""
Nov 04 01:46:42.741067 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.740627    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-f4d202\"/\"default-dockercfg-2hmgq\""
Nov 04 01:46:42.741067 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.740675    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-f4d202\"/\"kube-root-ca.crt\""
Nov 04 01:46:42.742955 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.742930    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"]
Nov 04 01:46:42.757792 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.757762    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-home\") pod \"simulator-w4-f4d202-kserve-849dc49777-7wdsk\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") " pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.757902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.757803    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsnwd\" (UniqueName: \"kubernetes.io/projected/a242d84a-50e6-4dbc-8179-60f694e0b80c-kube-api-access-dsnwd\") pod \"simulator-w4-f4d202-kserve-849dc49777-7wdsk\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") " pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.757902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.757836    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-model-cache\") pod \"simulator-w4-f4d202-kserve-849dc49777-7wdsk\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") " pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.757998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.757964    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-dshm\") pod \"simulator-w4-f4d202-kserve-849dc49777-7wdsk\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") " pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.758047 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.758019    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a242d84a-50e6-4dbc-8179-60f694e0b80c-tls-certs\") pod \"simulator-w4-f4d202-kserve-849dc49777-7wdsk\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") " pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.758132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.758112    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-tmp-dir\") pod \"simulator-w4-f4d202-kserve-849dc49777-7wdsk\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") " pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.858688 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.858655    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-home\") pod \"simulator-w4-f4d202-kserve-849dc49777-7wdsk\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") " pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.858860 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.858696    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsnwd\" (UniqueName: \"kubernetes.io/projected/a242d84a-50e6-4dbc-8179-60f694e0b80c-kube-api-access-dsnwd\") pod \"simulator-w4-f4d202-kserve-849dc49777-7wdsk\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") " pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.858860 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.858738    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-model-cache\") pod \"simulator-w4-f4d202-kserve-849dc49777-7wdsk\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") " pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.858860 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.858774    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-dshm\") pod \"simulator-w4-f4d202-kserve-849dc49777-7wdsk\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") " pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.858860 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.858821    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a242d84a-50e6-4dbc-8179-60f694e0b80c-tls-certs\") pod \"simulator-w4-f4d202-kserve-849dc49777-7wdsk\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") " pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.858860 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.858857    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-tmp-dir\") pod \"simulator-w4-f4d202-kserve-849dc49777-7wdsk\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") " pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.859117 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.859097    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-home\") pod \"simulator-w4-f4d202-kserve-849dc49777-7wdsk\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") " pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.859179 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.859142    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-model-cache\") pod \"simulator-w4-f4d202-kserve-849dc49777-7wdsk\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") " pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.859235 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.859222    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-tmp-dir\") pod \"simulator-w4-f4d202-kserve-849dc49777-7wdsk\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") " pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.861028 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.861003    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-dshm\") pod \"simulator-w4-f4d202-kserve-849dc49777-7wdsk\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") " pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.861321 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.861304    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a242d84a-50e6-4dbc-8179-60f694e0b80c-tls-certs\") pod \"simulator-w4-f4d202-kserve-849dc49777-7wdsk\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") " pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.920442 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.920410    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsnwd\" (UniqueName: \"kubernetes.io/projected/a242d84a-50e6-4dbc-8179-60f694e0b80c-kube-api-access-dsnwd\") pod \"simulator-w4-f4d202-kserve-849dc49777-7wdsk\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") " pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:42.952568 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:42.952536    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:43.329586 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:43.329559    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"]
Nov 04 01:46:43.330787 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:46:43.330760    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda242d84a_50e6_4dbc_8179_60f694e0b80c.slice/crio-cf7aa3dd69a944cd0602c610d4b4202b73691f17b8dbe7a61d658054532b965f WatchSource:0}: Error finding container cf7aa3dd69a944cd0602c610d4b4202b73691f17b8dbe7a61d658054532b965f: Status 404 returned error can't find the container with id cf7aa3dd69a944cd0602c610d4b4202b73691f17b8dbe7a61d658054532b965f
Nov 04 01:46:43.453536 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:43.453502    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk" event={"ID":"a242d84a-50e6-4dbc-8179-60f694e0b80c","Type":"ContainerStarted","Data":"043c9570f32c2e61141af943fb2994fb551e0c9384e3be3bb5e3cd47fa568ac1"}
Nov 04 01:46:43.453735 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:43.453547    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk" event={"ID":"a242d84a-50e6-4dbc-8179-60f694e0b80c","Type":"ContainerStarted","Data":"cf7aa3dd69a944cd0602c610d4b4202b73691f17b8dbe7a61d658054532b965f"}
Nov 04 01:46:43.572200 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:43.572144    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk" podStartSLOduration=1.572130766 podStartE2EDuration="1.572130766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:46:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:46:43.569033451 +0000 UTC m=+688.690310976" watchObservedRunningTime="2026-11-04 01:46:43.572130766 +0000 UTC m=+688.693408326"
Nov 04 01:46:43.952651 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:43.952608    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:48.526425 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:48.526393    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:46:52.952677 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:52.952644    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:52.965961 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:52.965933    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:46:53.519267 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:46:53.519237    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:47:06.748414 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:06.748374    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:47:11.990263 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:47:11.990225    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:47:17.185254 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.185212    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"]
Nov 04 01:47:17.190079 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.190062    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.268238 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.268205    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"]
Nov 04 01:47:17.294286 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.294256    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-fa0149\"/\"premium-w3-fa0149-kserve-self-signed-certs\""
Nov 04 01:47:17.302644 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.302620    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-tmp-dir\") pod \"premium-w3-fa0149-kserve-5678946dbd-fpvmr\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") " pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.302791 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.302650    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-home\") pod \"premium-w3-fa0149-kserve-5678946dbd-fpvmr\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") " pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.302791 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.302668    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-dshm\") pod \"premium-w3-fa0149-kserve-5678946dbd-fpvmr\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") " pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.302906 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.302850    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgrk2\" (UniqueName: \"kubernetes.io/projected/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-kube-api-access-kgrk2\") pod \"premium-w3-fa0149-kserve-5678946dbd-fpvmr\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") " pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.302955 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.302907    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-tls-certs\") pod \"premium-w3-fa0149-kserve-5678946dbd-fpvmr\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") " pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.302993 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.302950    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-model-cache\") pod \"premium-w3-fa0149-kserve-5678946dbd-fpvmr\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") " pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.404473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.404410    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgrk2\" (UniqueName: \"kubernetes.io/projected/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-kube-api-access-kgrk2\") pod \"premium-w3-fa0149-kserve-5678946dbd-fpvmr\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") " pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.404665 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.404487    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-tls-certs\") pod \"premium-w3-fa0149-kserve-5678946dbd-fpvmr\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") " pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.404665 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.404517    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-model-cache\") pod \"premium-w3-fa0149-kserve-5678946dbd-fpvmr\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") " pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.404665 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.404622    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-tmp-dir\") pod \"premium-w3-fa0149-kserve-5678946dbd-fpvmr\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") " pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.404665 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.404653    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-home\") pod \"premium-w3-fa0149-kserve-5678946dbd-fpvmr\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") " pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.404858 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.404679    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-dshm\") pod \"premium-w3-fa0149-kserve-5678946dbd-fpvmr\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") " pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.405063 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.405026    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-home\") pod \"premium-w3-fa0149-kserve-5678946dbd-fpvmr\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") " pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.405170 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.405069    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-model-cache\") pod \"premium-w3-fa0149-kserve-5678946dbd-fpvmr\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") " pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.405343 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.405317    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-tmp-dir\") pod \"premium-w3-fa0149-kserve-5678946dbd-fpvmr\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") " pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.407040 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.407022    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-dshm\") pod \"premium-w3-fa0149-kserve-5678946dbd-fpvmr\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") " pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.407293 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.407273    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-tls-certs\") pod \"premium-w3-fa0149-kserve-5678946dbd-fpvmr\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") " pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.465609 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.465530    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgrk2\" (UniqueName: \"kubernetes.io/projected/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-kube-api-access-kgrk2\") pod \"premium-w3-fa0149-kserve-5678946dbd-fpvmr\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") " pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.501738 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.501705    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:17.892712 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:17.892672    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"]
Nov 04 01:47:17.893840 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:47:17.893814    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fbf4e49_a222_4caf_b3d9_0fdbe663ebe7.slice/crio-6887f2852c73a9960725d3e5bfc91a24705098e853e5fd1d5f35d0195e71a479 WatchSource:0}: Error finding container 6887f2852c73a9960725d3e5bfc91a24705098e853e5fd1d5f35d0195e71a479: Status 404 returned error can't find the container with id 6887f2852c73a9960725d3e5bfc91a24705098e853e5fd1d5f35d0195e71a479
Nov 04 01:47:18.653325 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:18.653287    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr" event={"ID":"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7","Type":"ContainerStarted","Data":"ad52477463364614b7653bd9dde4b90ea67fe644b8f66e8de94d61f2dbfa94cf"}
Nov 04 01:47:18.653325 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:18.653329    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr" event={"ID":"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7","Type":"ContainerStarted","Data":"6887f2852c73a9960725d3e5bfc91a24705098e853e5fd1d5f35d0195e71a479"}
Nov 04 01:47:18.770815 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:18.770756    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr" podStartSLOduration=1.770740386 podStartE2EDuration="1.770740386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:47:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:47:18.769226445 +0000 UTC m=+723.890503971" watchObservedRunningTime="2026-11-04 01:47:18.770740386 +0000 UTC m=+723.892017912"
Nov 04 01:47:19.501845 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:19.501810    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:21.840688 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:21.840653    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:47:27.501961 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:27.501879    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:27.516620 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:27.516593    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:27.716370 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:27.716340    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:47:38.128052 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.128014    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"]
Nov 04 01:47:38.131857 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.131840    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.184078 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.184050    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-7af38e\"/\"openshift-service-ca.crt\""
Nov 04 01:47:38.220624 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.220588    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb2sf\" (UniqueName: \"kubernetes.io/projected/a1501413-7507-4f1d-ae16-de82aed103ea-kube-api-access-zb2sf\") pod \"premium-w0-7af38e-kserve-75685b786b-xmcwj\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") " pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.220802 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.220635    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-dshm\") pod \"premium-w0-7af38e-kserve-75685b786b-xmcwj\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") " pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.220802 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.220704    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1501413-7507-4f1d-ae16-de82aed103ea-tls-certs\") pod \"premium-w0-7af38e-kserve-75685b786b-xmcwj\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") " pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.220882 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.220848    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-home\") pod \"premium-w0-7af38e-kserve-75685b786b-xmcwj\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") " pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.220915 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.220890    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-model-cache\") pod \"premium-w0-7af38e-kserve-75685b786b-xmcwj\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") " pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.220969 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.220958    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-tmp-dir\") pod \"premium-w0-7af38e-kserve-75685b786b-xmcwj\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") " pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.233666 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.233641    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-7af38e\"/\"kube-root-ca.crt\""
Nov 04 01:47:38.233991 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.233962    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-7af38e\"/\"premium-w0-7af38e-kserve-self-signed-certs\""
Nov 04 01:47:38.234112 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.233999    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-7af38e\"/\"default-dockercfg-7m2lf\""
Nov 04 01:47:38.236390 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.236368    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"]
Nov 04 01:47:38.321987 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.321953    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-home\") pod \"premium-w0-7af38e-kserve-75685b786b-xmcwj\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") " pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.322159 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.321994    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-model-cache\") pod \"premium-w0-7af38e-kserve-75685b786b-xmcwj\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") " pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.322159 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.322039    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-tmp-dir\") pod \"premium-w0-7af38e-kserve-75685b786b-xmcwj\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") " pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.322159 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.322081    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb2sf\" (UniqueName: \"kubernetes.io/projected/a1501413-7507-4f1d-ae16-de82aed103ea-kube-api-access-zb2sf\") pod \"premium-w0-7af38e-kserve-75685b786b-xmcwj\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") " pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.322159 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.322118    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-dshm\") pod \"premium-w0-7af38e-kserve-75685b786b-xmcwj\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") " pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.322321 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.322159    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1501413-7507-4f1d-ae16-de82aed103ea-tls-certs\") pod \"premium-w0-7af38e-kserve-75685b786b-xmcwj\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") " pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.322411 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.322391    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-home\") pod \"premium-w0-7af38e-kserve-75685b786b-xmcwj\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") " pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.322539 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.322520    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-tmp-dir\") pod \"premium-w0-7af38e-kserve-75685b786b-xmcwj\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") " pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.322616 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.322577    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-model-cache\") pod \"premium-w0-7af38e-kserve-75685b786b-xmcwj\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") " pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.325387 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.325359    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-dshm\") pod \"premium-w0-7af38e-kserve-75685b786b-xmcwj\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") " pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.325721 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.325703    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1501413-7507-4f1d-ae16-de82aed103ea-tls-certs\") pod \"premium-w0-7af38e-kserve-75685b786b-xmcwj\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") " pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.382881 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.382814    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb2sf\" (UniqueName: \"kubernetes.io/projected/a1501413-7507-4f1d-ae16-de82aed103ea-kube-api-access-zb2sf\") pod \"premium-w0-7af38e-kserve-75685b786b-xmcwj\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") " pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.443646 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.443614    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:38.831847 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:38.831808    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"]
Nov 04 01:47:38.832863 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:47:38.832837    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1501413_7507_4f1d_ae16_de82aed103ea.slice/crio-49a306715af304c202b1a6de5b150c7eb5b3f477802ab8b35043c06c0b5a5873 WatchSource:0}: Error finding container 49a306715af304c202b1a6de5b150c7eb5b3f477802ab8b35043c06c0b5a5873: Status 404 returned error can't find the container with id 49a306715af304c202b1a6de5b150c7eb5b3f477802ab8b35043c06c0b5a5873
Nov 04 01:47:39.776047 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:39.776007    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj" event={"ID":"a1501413-7507-4f1d-ae16-de82aed103ea","Type":"ContainerStarted","Data":"fafa4c495d103497209c7a0f4bce8f4d17dd6db09e34118228c8ab04e95ffe52"}
Nov 04 01:47:39.776047 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:39.776045    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj" event={"ID":"a1501413-7507-4f1d-ae16-de82aed103ea","Type":"ContainerStarted","Data":"49a306715af304c202b1a6de5b150c7eb5b3f477802ab8b35043c06c0b5a5873"}
Nov 04 01:47:39.894716 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:39.894663    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj" podStartSLOduration=1.894647813 podStartE2EDuration="1.894647813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:47:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:47:39.891894239 +0000 UTC m=+745.013171764" watchObservedRunningTime="2026-11-04 01:47:39.894647813 +0000 UTC m=+745.015925338"
Nov 04 01:47:40.444020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:40.443987    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:41.991563 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:47:41.991526    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:47:42.911597 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:42.911560    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:47:48.444723 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:48.444694    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:48.457062 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:48.457036    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:47:48.845584 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:47:48.845506    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:48:00.627506 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.627469    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"]
Nov 04 01:48:00.631250 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.631233    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.736240 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.736209    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-795045\"/\"openshift-service-ca.crt\""
Nov 04 01:48:00.736407 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.736348    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-795045\"/\"kube-root-ca.crt\""
Nov 04 01:48:00.736543 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.736512    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-795045\"/\"premium-w1-795045-kserve-self-signed-certs\""
Nov 04 01:48:00.736543 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.736524    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-795045\"/\"default-dockercfg-8hpdq\""
Nov 04 01:48:00.738717 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.738699    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"]
Nov 04 01:48:00.765230 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.765200    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-dshm\") pod \"premium-w1-795045-kserve-86d9bc5fc5-ffhvl\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") " pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.765386 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.765254    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-model-cache\") pod \"premium-w1-795045-kserve-86d9bc5fc5-ffhvl\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") " pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.765386 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.765350    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfvp5\" (UniqueName: \"kubernetes.io/projected/b42d0649-0316-4a47-af9f-e271c5b7068f-kube-api-access-bfvp5\") pod \"premium-w1-795045-kserve-86d9bc5fc5-ffhvl\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") " pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.765524 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.765401    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-home\") pod \"premium-w1-795045-kserve-86d9bc5fc5-ffhvl\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") " pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.765565 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.765545    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-tmp-dir\") pod \"premium-w1-795045-kserve-86d9bc5fc5-ffhvl\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") " pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.765603 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.765581    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b42d0649-0316-4a47-af9f-e271c5b7068f-tls-certs\") pod \"premium-w1-795045-kserve-86d9bc5fc5-ffhvl\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") " pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.866156 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.866120    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-tmp-dir\") pod \"premium-w1-795045-kserve-86d9bc5fc5-ffhvl\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") " pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.866156 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.866162    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b42d0649-0316-4a47-af9f-e271c5b7068f-tls-certs\") pod \"premium-w1-795045-kserve-86d9bc5fc5-ffhvl\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") " pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.866394 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.866193    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-dshm\") pod \"premium-w1-795045-kserve-86d9bc5fc5-ffhvl\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") " pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.866394 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.866228    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-model-cache\") pod \"premium-w1-795045-kserve-86d9bc5fc5-ffhvl\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") " pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.866394 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.866256    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfvp5\" (UniqueName: \"kubernetes.io/projected/b42d0649-0316-4a47-af9f-e271c5b7068f-kube-api-access-bfvp5\") pod \"premium-w1-795045-kserve-86d9bc5fc5-ffhvl\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") " pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.866394 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.866347    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-home\") pod \"premium-w1-795045-kserve-86d9bc5fc5-ffhvl\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") " pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.866644 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.866614    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-tmp-dir\") pod \"premium-w1-795045-kserve-86d9bc5fc5-ffhvl\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") " pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.866696 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.866651    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-model-cache\") pod \"premium-w1-795045-kserve-86d9bc5fc5-ffhvl\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") " pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.866797 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.866779    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-home\") pod \"premium-w1-795045-kserve-86d9bc5fc5-ffhvl\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") " pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.868732 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.868709    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-dshm\") pod \"premium-w1-795045-kserve-86d9bc5fc5-ffhvl\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") " pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.868965 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.868948    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b42d0649-0316-4a47-af9f-e271c5b7068f-tls-certs\") pod \"premium-w1-795045-kserve-86d9bc5fc5-ffhvl\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") " pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.927638 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.927607    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfvp5\" (UniqueName: \"kubernetes.io/projected/b42d0649-0316-4a47-af9f-e271c5b7068f-kube-api-access-bfvp5\") pod \"premium-w1-795045-kserve-86d9bc5fc5-ffhvl\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") " pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:00.943698 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:00.943667    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:01.333981 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:01.333958    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"]
Nov 04 01:48:01.335492 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:48:01.335462    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb42d0649_0316_4a47_af9f_e271c5b7068f.slice/crio-69d874f1595d82f7c8ddd0b60a657d7d58cb6c378a0270cf453bd17707e4454e WatchSource:0}: Error finding container 69d874f1595d82f7c8ddd0b60a657d7d58cb6c378a0270cf453bd17707e4454e: Status 404 returned error can't find the container with id 69d874f1595d82f7c8ddd0b60a657d7d58cb6c378a0270cf453bd17707e4454e
Nov 04 01:48:01.919854 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:01.919802    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl" event={"ID":"b42d0649-0316-4a47-af9f-e271c5b7068f","Type":"ContainerStarted","Data":"933a0891f43c9126aa89364737c99eeac432943022a60f7a6708b38c66abb15d"}
Nov 04 01:48:01.920322 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:01.919860    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl" event={"ID":"b42d0649-0316-4a47-af9f-e271c5b7068f","Type":"ContainerStarted","Data":"69d874f1595d82f7c8ddd0b60a657d7d58cb6c378a0270cf453bd17707e4454e"}
Nov 04 01:48:01.944073 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:01.944040    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:02.038286 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:02.038228    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl" podStartSLOduration=2.038213044 podStartE2EDuration="2.038213044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:48:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:48:02.03754975 +0000 UTC m=+767.158827273" watchObservedRunningTime="2026-11-04 01:48:02.038213044 +0000 UTC m=+767.159490569"
Nov 04 01:48:07.994855 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:07.994816    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:48:10.944436 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:10.944394    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:10.957070 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:10.957041    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:10.981390 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:10.981361    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:48:11.992262 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:48:11.992224    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:48:31.738165 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.738104    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"]
Nov 04 01:48:31.742144 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.742123    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.788457 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.788418    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-istio-data\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.788664 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.788516    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-workload-socket\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.788664 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.788571    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-workload-certs\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.789064 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.789039    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhlc9\" (UniqueName: \"kubernetes.io/projected/4229f224-e59f-4caa-9837-f303c79c72a2-kube-api-access-zhlc9\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.789641 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.789083    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4229f224-e59f-4caa-9837-f303c79c72a2-istiod-ca-cert\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.789641 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.789149    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-credential-socket\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.789641 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.789180    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4229f224-e59f-4caa-9837-f303c79c72a2-istio-podinfo\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.789641 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.789220    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4229f224-e59f-4caa-9837-f303c79c72a2-istio-token\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.789641 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.789247    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-istio-envoy\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.795383 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.794835    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-795045-openshift-default-dockercfg-bm9hd\""
Nov 04 01:48:31.847397 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.847365    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"]
Nov 04 01:48:31.890761 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.890663    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhlc9\" (UniqueName: \"kubernetes.io/projected/4229f224-e59f-4caa-9837-f303c79c72a2-kube-api-access-zhlc9\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.890952 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.890796    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4229f224-e59f-4caa-9837-f303c79c72a2-istiod-ca-cert\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.890952 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.890846    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-credential-socket\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.890952 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.890870    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4229f224-e59f-4caa-9837-f303c79c72a2-istio-podinfo\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.890952 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.890899    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4229f224-e59f-4caa-9837-f303c79c72a2-istio-token\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.890952 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.890916    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-istio-envoy\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.891195 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.890988    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-istio-data\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.891195 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.891030    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-workload-socket\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.891195 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.891099    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-workload-certs\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.891393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.891363    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-credential-socket\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.891553 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.891411    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-istio-data\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.891553 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.891496    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-workload-socket\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.891749 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.891723    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4229f224-e59f-4caa-9837-f303c79c72a2-istiod-ca-cert\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.891828 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.891774    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-workload-certs\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.893815 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.893788    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-istio-envoy\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.893989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.893972    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4229f224-e59f-4caa-9837-f303c79c72a2-istio-podinfo\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.952199 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.952172    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4229f224-e59f-4caa-9837-f303c79c72a2-istio-token\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:31.952592 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:31.952573    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhlc9\" (UniqueName: \"kubernetes.io/projected/4229f224-e59f-4caa-9837-f303c79c72a2-kube-api-access-zhlc9\") pod \"e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") " pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:32.054930 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:32.054849    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:32.485689 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:32.485661    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"]
Nov 04 01:48:32.486806 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:48:32.486783    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4229f224_e59f_4caa_9837_f303c79c72a2.slice/crio-822754703b7438159664fdae801724b47db2d92a6f0f329f204bf01af4b0a198 WatchSource:0}: Error finding container 822754703b7438159664fdae801724b47db2d92a6f0f329f204bf01af4b0a198: Status 404 returned error can't find the container with id 822754703b7438159664fdae801724b47db2d92a6f0f329f204bf01af4b0a198
Nov 04 01:48:32.489146 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:32.489111    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:48:32.489244 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:32.489195    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:48:32.489244 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:32.489239    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:48:33.102253 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:33.102208    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz" event={"ID":"4229f224-e59f-4caa-9837-f303c79c72a2","Type":"ContainerStarted","Data":"6575347f728cf6f78a12bb0c44e1c6459b10370f3c76a2e15d3d0d0717c1e1b1"}
Nov 04 01:48:33.102753 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:33.102267    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz" event={"ID":"4229f224-e59f-4caa-9837-f303c79c72a2","Type":"ContainerStarted","Data":"822754703b7438159664fdae801724b47db2d92a6f0f329f204bf01af4b0a198"}
Nov 04 01:48:33.224972 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:33.224920    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz" podStartSLOduration=2.224907521 podStartE2EDuration="2.224907521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:48:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:48:33.223601655 +0000 UTC m=+798.344879181" watchObservedRunningTime="2026-11-04 01:48:33.224907521 +0000 UTC m=+798.346185047"
Nov 04 01:48:34.067473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:34.064072    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:34.070655 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:34.070615    6116 patch_prober.go:29] interesting pod/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Nov 04 01:48:34.070849 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:34.070686    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz" podUID="4229f224-e59f-4caa-9837-f303c79c72a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Nov 04 01:48:34.542827 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:34.542791    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:48:35.055893 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:35.055855    6116 patch_prober.go:29] interesting pod/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Nov 04 01:48:35.056181 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:35.056154    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz" podUID="4229f224-e59f-4caa-9837-f303c79c72a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Nov 04 01:48:36.070711 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:36.070653    6116 patch_prober.go:29] interesting pod/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Nov 04 01:48:36.071192 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:36.070752    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz" podUID="4229f224-e59f-4caa-9837-f303c79c72a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Nov 04 01:48:37.066481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:37.057895    6116 patch_prober.go:29] interesting pod/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Nov 04 01:48:37.066481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:37.057957    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz" podUID="4229f224-e59f-4caa-9837-f303c79c72a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Nov 04 01:48:38.055953 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:38.055914    6116 patch_prober.go:29] interesting pod/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Nov 04 01:48:38.056610 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:38.056576    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz" podUID="4229f224-e59f-4caa-9837-f303c79c72a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Nov 04 01:48:39.055881 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:39.055841    6116 patch_prober.go:29] interesting pod/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Nov 04 01:48:39.056092 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:39.055928    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz" podUID="4229f224-e59f-4caa-9837-f303c79c72a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Nov 04 01:48:40.288920 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:40.288888    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:40.289508 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:40.289480    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:40.290362 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:48:40.290344    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:48:41.991494 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:48:41.991442    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:49:04.907691 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:04.907652    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:49:11.991912 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:49:11.991865    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:49:24.728320 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.728289    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"]
Nov 04 01:49:24.732645 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.732627    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.785458 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.785416    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-fa0149-openshift-default-dockercfg-9mtst\""
Nov 04 01:49:24.811527 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.811493    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-credential-socket\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.811694 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.811558    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-workload-socket\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.811694 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.811661    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-data\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.812029 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.811705    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-podinfo\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.812029 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.811769    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-token\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.812029 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.811816    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44zkb\" (UniqueName: \"kubernetes.io/projected/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-kube-api-access-44zkb\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.812029 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.811877    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istiod-ca-cert\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.812029 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.811897    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-envoy\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.812029 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.811914    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-workload-certs\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.838348 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.838319    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"]
Nov 04 01:49:24.913439 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.913403    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-credential-socket\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.913662 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.913537    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-workload-socket\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.913662 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.913615    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-data\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.913769 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.913671    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-podinfo\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.913769 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.913692    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-token\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.913769 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.913737    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44zkb\" (UniqueName: \"kubernetes.io/projected/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-kube-api-access-44zkb\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.913889 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.913782    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istiod-ca-cert\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.913889 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.913811    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-envoy\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.913889 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.913812    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-credential-socket\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.913889 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.913839    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-workload-certs\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.914053 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.913927    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-data\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.914165 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.914144    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-workload-certs\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.914267 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.914238    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-workload-socket\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.914633 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.914425    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istiod-ca-cert\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.916081 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.916061    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-envoy\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.916404 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.916382    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-podinfo\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.975033 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.975004    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-token\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:24.975192 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:24.975129    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44zkb\" (UniqueName: \"kubernetes.io/projected/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-kube-api-access-44zkb\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:25.046101 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:25.046008    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:25.473865 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:25.473833    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"]
Nov 04 01:49:25.474679 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:49:25.474647    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e55d400_e27f_45fd_b79c_50f9ca6c7da3.slice/crio-5407e7f33f35e088352a20f038a609280a5aa081a4cb081596a0741baba473dc WatchSource:0}: Error finding container 5407e7f33f35e088352a20f038a609280a5aa081a4cb081596a0741baba473dc: Status 404 returned error can't find the container with id 5407e7f33f35e088352a20f038a609280a5aa081a4cb081596a0741baba473dc
Nov 04 01:49:25.476982 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:25.476946    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:49:25.477050 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:25.477029    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:49:25.477088 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:25.477079    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:49:26.414461 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:26.414420    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4" event={"ID":"5e55d400-e27f-45fd-b79c-50f9ca6c7da3","Type":"ContainerStarted","Data":"f77f96fd3f72210521c3ea594e3ee6060df08975b375279ca59065d18af02886"}
Nov 04 01:49:26.414974 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:26.414499    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4" event={"ID":"5e55d400-e27f-45fd-b79c-50f9ca6c7da3","Type":"ContainerStarted","Data":"5407e7f33f35e088352a20f038a609280a5aa081a4cb081596a0741baba473dc"}
Nov 04 01:49:26.610113 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:26.610032    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4" podStartSLOduration=2.610013531 podStartE2EDuration="2.610013531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:49:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:49:26.608919983 +0000 UTC m=+851.730197508" watchObservedRunningTime="2026-11-04 01:49:26.610013531 +0000 UTC m=+851.731291059"
Nov 04 01:49:27.070661 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:27.070610    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:27.072723 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:27.072693    6116 patch_prober.go:29] interesting pod/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 04 01:49:27.072866 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:27.072746    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4" podUID="5e55d400-e27f-45fd-b79c-50f9ca6c7da3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 04 01:49:28.063480 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:28.059479    6116 patch_prober.go:29] interesting pod/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 04 01:49:28.063480 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:28.059538    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4" podUID="5e55d400-e27f-45fd-b79c-50f9ca6c7da3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 04 01:49:29.066485 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:29.066213    6116 patch_prober.go:29] interesting pod/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 04 01:49:29.066485 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:29.066283    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4" podUID="5e55d400-e27f-45fd-b79c-50f9ca6c7da3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 04 01:49:30.070614 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:30.070573    6116 patch_prober.go:29] interesting pod/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 04 01:49:30.071073 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:30.070645    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4" podUID="5e55d400-e27f-45fd-b79c-50f9ca6c7da3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 04 01:49:31.067476 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:31.066925    6116 patch_prober.go:29] interesting pod/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 04 01:49:31.067476 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:31.066991    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4" podUID="5e55d400-e27f-45fd-b79c-50f9ca6c7da3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 04 01:49:32.325274 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:32.325238    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:32.326126 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:32.326106    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:32.326791 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:32.326769    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:49:36.970523 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:36.970442    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"]
Nov 04 01:49:36.971050 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:36.970894    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55" podUID="98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" containerName="istio-proxy" containerID="cri-o://b6cd75338d8a842847708038fc8e80d38a59e7261514694172919298514e132b" gracePeriod=30
Nov 04 01:49:41.995109 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:49:41.995075    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:49:42.236057 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.235990    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:49:42.343999 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.343965    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-podinfo\") pod \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") "
Nov 04 01:49:42.344199 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.344044    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-envoy\") pod \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") "
Nov 04 01:49:42.344199 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.344078    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-kube-api-access-pzhp4\" (UniqueName: \"kubernetes.io/projected/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-kube-api-access-pzhp4\") pod \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") "
Nov 04 01:49:42.344199 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.344129    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-data\") pod \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") "
Nov 04 01:49:42.344361 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.344256    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-credential-socket\") pod \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") "
Nov 04 01:49:42.344361 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.344313    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-workload-socket\") pod \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") "
Nov 04 01:49:42.344361 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.344352    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-token\" (UniqueName: \"kubernetes.io/projected/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-token\") pod \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") "
Nov 04 01:49:42.344548 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.344409    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-data" pod "98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" (UID: "98c2fe44-752b-4134-8e4d-d4a3db4cdf5f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:49:42.344548 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.344440    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istiod-ca-cert\") pod \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") "
Nov 04 01:49:42.344654 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.344593    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-workload-certs\") pod \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\" (UID: \"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f\") "
Nov 04 01:49:42.344695 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.344648    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-workload-socket" pod "98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" (UID: "98c2fe44-752b-4134-8e4d-d4a3db4cdf5f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:49:42.345358 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.345218    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:49:42.345358 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.345249    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:49:42.345358 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.345289    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istiod-ca-cert" pod "98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" (UID: "98c2fe44-752b-4134-8e4d-d4a3db4cdf5f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:49:42.345647 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.345357    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-workload-certs" pod "98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" (UID: "98c2fe44-752b-4134-8e4d-d4a3db4cdf5f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:49:42.345647 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.345595    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-credential-socket" pod "98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" (UID: "98c2fe44-752b-4134-8e4d-d4a3db4cdf5f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:49:42.346575 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.346550    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-envoy" pod "98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" (UID: "98c2fe44-752b-4134-8e4d-d4a3db4cdf5f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:49:42.346719 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.346700    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-podinfo" pod "98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" (UID: "98c2fe44-752b-4134-8e4d-d4a3db4cdf5f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:49:42.346782 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.346720    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-kube-api-access-pzhp4" pod "98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" (UID: "98c2fe44-752b-4134-8e4d-d4a3db4cdf5f"). InnerVolumeSpecName "kube-api-access-pzhp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:49:42.346836 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.346823    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-token" pod "98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" (UID: "98c2fe44-752b-4134-8e4d-d4a3db4cdf5f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:49:42.446698 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.446626    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:49:42.446698 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.446696    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:49:42.446912 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.446709    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzhp4\" (UniqueName: \"kubernetes.io/projected/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-kube-api-access-pzhp4\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:49:42.446912 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.446718    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:49:42.446912 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.446727    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:49:42.446912 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.446736    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:49:42.446912 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.446745    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:49:42.511809 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.511722    6116 generic.go:356] "Generic (PLEG): container finished" podID="98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" containerID="b6cd75338d8a842847708038fc8e80d38a59e7261514694172919298514e132b" exitCode=0
Nov 04 01:49:42.511809 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.511781    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"
Nov 04 01:49:42.512008 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.511807    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55" event={"ID":"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f","Type":"ContainerDied","Data":"b6cd75338d8a842847708038fc8e80d38a59e7261514694172919298514e132b"}
Nov 04 01:49:42.512008 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.511873    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55" event={"ID":"98c2fe44-752b-4134-8e4d-d4a3db4cdf5f","Type":"ContainerDied","Data":"a9b2485a2c36222c093a0a702b3fa05ed9b537d893fa662c3bc00480348bb1c9"}
Nov 04 01:49:42.512008 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.511901    6116 scope.go:122] "RemoveContainer" containerID="b6cd75338d8a842847708038fc8e80d38a59e7261514694172919298514e132b"
Nov 04 01:49:42.523945 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.523929    6116 scope.go:122] "RemoveContainer" containerID="b6cd75338d8a842847708038fc8e80d38a59e7261514694172919298514e132b"
Nov 04 01:49:42.524233 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:49:42.524215    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6cd75338d8a842847708038fc8e80d38a59e7261514694172919298514e132b\": container with ID starting with b6cd75338d8a842847708038fc8e80d38a59e7261514694172919298514e132b not found: ID does not exist" containerID="b6cd75338d8a842847708038fc8e80d38a59e7261514694172919298514e132b"
Nov 04 01:49:42.524282 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.524242    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6cd75338d8a842847708038fc8e80d38a59e7261514694172919298514e132b"} err="failed to get container status \"b6cd75338d8a842847708038fc8e80d38a59e7261514694172919298514e132b\": rpc error: code = NotFound desc = could not find container \"b6cd75338d8a842847708038fc8e80d38a59e7261514694172919298514e132b\": container with ID starting with b6cd75338d8a842847708038fc8e80d38a59e7261514694172919298514e132b not found: ID does not exist"
Nov 04 01:49:42.677742 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.677707    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"]
Nov 04 01:49:42.680551 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:42.680524    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55"]
Nov 04 01:49:43.027430 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:43.027397    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" path="/var/lib/kubelet/pods/98c2fe44-752b-4134-8e4d-d4a3db4cdf5f/volumes"
Nov 04 01:49:53.545243 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:49:53.545205    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:50:11.992037 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:50:11.992002    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:50:15.195771 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.195737    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/prometheus/0.log"
Nov 04 01:50:15.196202 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.196043    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/prometheus/0.log"
Nov 04 01:50:15.208051 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.208026    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6lfbv_192e2fec-fefb-4bc5-9516-ed7464963a10/ovn-acl-logging/0.log"
Nov 04 01:50:15.208575 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.208554    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6lfbv_192e2fec-fefb-4bc5-9516-ed7464963a10/ovn-acl-logging/0.log"
Nov 04 01:50:15.353974 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.353940    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"]
Nov 04 01:50:15.354526 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.354510    6116 cpu_manager.go:414] "containerMap: removing container" podUID="98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" containerName="istio-proxy"
Nov 04 01:50:15.354526 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.354527    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" containerName="istio-proxy"
Nov 04 01:50:15.354694 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.354638    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="98c2fe44-752b-4134-8e4d-d4a3db4cdf5f" containerName="istio-proxy"
Nov 04 01:50:15.359165 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.359149    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.445328 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.445240    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-dshm\") pod \"distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.445328 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.445289    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-home\") pod \"distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.445571 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.445441    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8prbq\" (UniqueName: \"kubernetes.io/projected/a9f371eb-274b-42c2-ac99-83ba56e24600-kube-api-access-8prbq\") pod \"distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.445571 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.445498    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f371eb-274b-42c2-ac99-83ba56e24600-tls-certs\") pod \"distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.445649 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.445614    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-model-cache\") pod \"distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.445694 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.445645    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-tmp-dir\") pod \"distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.461226 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.461202    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-fa0149\"/\"distinct-2-w3-fa0149-kserve-self-signed-certs\""
Nov 04 01:50:15.463725 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.463701    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"]
Nov 04 01:50:15.546685 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.546653    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f371eb-274b-42c2-ac99-83ba56e24600-tls-certs\") pod \"distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.546883 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.546742    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-model-cache\") pod \"distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.546883 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.546762    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-tmp-dir\") pod \"distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.546883 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.546781    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-dshm\") pod \"distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.546883 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.546808    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-home\") pod \"distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.547074 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.546900    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8prbq\" (UniqueName: \"kubernetes.io/projected/a9f371eb-274b-42c2-ac99-83ba56e24600-kube-api-access-8prbq\") pod \"distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.547295 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.547268    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-tmp-dir\") pod \"distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.547403 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.547302    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-model-cache\") pod \"distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.547403 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.547338    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-home\") pod \"distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.549130 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.549109    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-dshm\") pod \"distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.549368 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.549351    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f371eb-274b-42c2-ac99-83ba56e24600-tls-certs\") pod \"distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.607580 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.607547    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8prbq\" (UniqueName: \"kubernetes.io/projected/a9f371eb-274b-42c2-ac99-83ba56e24600-kube-api-access-8prbq\") pod \"distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") " pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:15.671069 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:15.671032    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:16.048125 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:16.048096    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"]
Nov 04 01:50:16.049776 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:50:16.049745    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9f371eb_274b_42c2_ac99_83ba56e24600.slice/crio-7b115b84b0a10fdb7ad756704855bd54f9ad62174aacfab7ba7f70958fcd6589 WatchSource:0}: Error finding container 7b115b84b0a10fdb7ad756704855bd54f9ad62174aacfab7ba7f70958fcd6589: Status 404 returned error can't find the container with id 7b115b84b0a10fdb7ad756704855bd54f9ad62174aacfab7ba7f70958fcd6589
Nov 04 01:50:16.174110 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:16.174077    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:50:16.718634 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:16.718599    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk" event={"ID":"a9f371eb-274b-42c2-ac99-83ba56e24600","Type":"ContainerStarted","Data":"0d0f3e0fa5adff59bb5e810d90633148ad38b785884188dc0fa9f5a33d643a69"}
Nov 04 01:50:16.719024 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:16.718641    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk" event={"ID":"a9f371eb-274b-42c2-ac99-83ba56e24600","Type":"ContainerStarted","Data":"7b115b84b0a10fdb7ad756704855bd54f9ad62174aacfab7ba7f70958fcd6589"}
Nov 04 01:50:16.837016 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:16.836955    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk" podStartSLOduration=1.836936503 podStartE2EDuration="1.836936503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:50:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:50:16.835840129 +0000 UTC m=+901.957117655" watchObservedRunningTime="2026-11-04 01:50:16.836936503 +0000 UTC m=+901.958214028"
Nov 04 01:50:17.672159 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:17.672124    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:25.672282 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:25.672178    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:25.685055 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:25.685029    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:25.784943 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:25.784913    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:50:32.526759 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:32.526727    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:50:41.991686 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:50:41.991652    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:50:42.423829 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.423793    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"]
Nov 04 01:50:42.432696 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.432675    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.485074 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.485041    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-06099f-openshift-default-dockercfg-c974p\""
Nov 04 01:50:42.537391 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.537359    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"]
Nov 04 01:50:42.561481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.556185    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-podinfo\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.561481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.556274    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-workload-certs\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.561481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.556303    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-envoy\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.561481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.556361    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-credential-socket\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.561481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.556400    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqvhs\" (UniqueName: \"kubernetes.io/projected/35deac99-14d0-416d-9c8d-c8abb03f5f7d-kube-api-access-tqvhs\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.561481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.556462    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istiod-ca-cert\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.561481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.556521    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-data\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.561481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.556584    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-workload-socket\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.561481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.556658    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-token\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.657951 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.657909    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-podinfo\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.658106 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.657961    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-workload-certs\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.658106 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.658070    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-envoy\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.658209 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.658149    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-credential-socket\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.658209 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.658190    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqvhs\" (UniqueName: \"kubernetes.io/projected/35deac99-14d0-416d-9c8d-c8abb03f5f7d-kube-api-access-tqvhs\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.658309 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.658219    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istiod-ca-cert\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.658309 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.658282    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-data\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.658309 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.658295    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-workload-certs\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.658497 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.658365    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-workload-socket\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.658497 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.658489    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-token\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.658610 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.658549    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-credential-socket\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.658610 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.658596    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-data\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.658971 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.658947    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-workload-socket\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.659021 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.658958    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istiod-ca-cert\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.660405 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.660385    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-envoy\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.660680 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.660661    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-podinfo\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.719350 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.719274    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-token\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.719527 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.719407    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqvhs\" (UniqueName: \"kubernetes.io/projected/35deac99-14d0-416d-9c8d-c8abb03f5f7d-kube-api-access-tqvhs\") pod \"e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") " pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:42.746681 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:42.746650    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:43.173960 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:43.173936    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"]
Nov 04 01:50:43.175709 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:50:43.175684    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35deac99_14d0_416d_9c8d_c8abb03f5f7d.slice/crio-e9a962d572b0f1a463173acb92dd3b236809b310633ff2992dc9002290a7a19a WatchSource:0}: Error finding container e9a962d572b0f1a463173acb92dd3b236809b310633ff2992dc9002290a7a19a: Status 404 returned error can't find the container with id e9a962d572b0f1a463173acb92dd3b236809b310633ff2992dc9002290a7a19a
Nov 04 01:50:43.177923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:43.177891    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:50:43.177988 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:43.177956    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:50:43.177988 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:43.177984    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:50:43.883869 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:43.883835    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6" event={"ID":"35deac99-14d0-416d-9c8d-c8abb03f5f7d","Type":"ContainerStarted","Data":"ef2cd128dacca8851fb28b42841f1f650dfc53652a7704966971dae8a36b09de"}
Nov 04 01:50:43.884053 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:43.883895    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6" event={"ID":"35deac99-14d0-416d-9c8d-c8abb03f5f7d","Type":"ContainerStarted","Data":"e9a962d572b0f1a463173acb92dd3b236809b310633ff2992dc9002290a7a19a"}
Nov 04 01:50:44.005899 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:44.005838    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6" podStartSLOduration=2.005823252 podStartE2EDuration="2.005823252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:50:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:50:44.003930414 +0000 UTC m=+929.125207963" watchObservedRunningTime="2026-11-04 01:50:44.005823252 +0000 UTC m=+929.127100770"
Nov 04 01:50:44.747373 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:44.747337    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:44.752863 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:44.752836    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:44.889575 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:44.889539    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:50:44.890548 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:50:44.890512    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:51:04.481516 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:04.481475    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:51:10.193608 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.193574    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"]
Nov 04 01:51:10.197706 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.197685    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.287184 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.287145    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.287357 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.287201    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.287357 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.287269    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.287357 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.287302    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bd672da-f401-43ae-a6bd-52247191c3a9-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.287515 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.287391    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ktmz\" (UniqueName: \"kubernetes.io/projected/1bd672da-f401-43ae-a6bd-52247191c3a9-kube-api-access-5ktmz\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.287515 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.287442    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.287597 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.287543    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.287597 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.287582    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.287668 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.287608    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.302622 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.302584    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"]
Nov 04 01:51:10.389234 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.389197    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.389234 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.389241    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.389523 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.389262    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.389523 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.389282    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bd672da-f401-43ae-a6bd-52247191c3a9-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.389523 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.389319    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ktmz\" (UniqueName: \"kubernetes.io/projected/1bd672da-f401-43ae-a6bd-52247191c3a9-kube-api-access-5ktmz\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.389523 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.389340    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.389523 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.389383    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.389523 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.389420    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.389523 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.389461    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.389849 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.389668    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.389849 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.389695    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.389849 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.389775    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.389962 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.389896    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.390138 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.390119    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bd672da-f401-43ae-a6bd-52247191c3a9-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.391713 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.391690    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.392180 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.392161    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.450213 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.450142    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.450343 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.450272    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ktmz\" (UniqueName: \"kubernetes.io/projected/1bd672da-f401-43ae-a6bd-52247191c3a9-kube-api-access-5ktmz\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.511406 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.511374    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:10.938224 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.938198    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"]
Nov 04 01:51:10.940034 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:51:10.940005    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bd672da_f401_43ae_a6bd_52247191c3a9.slice/crio-33f78a88170cedf84a5ba2b4a3b5c9ef0188aa57910f96e16961de259787f6e5 WatchSource:0}: Error finding container 33f78a88170cedf84a5ba2b4a3b5c9ef0188aa57910f96e16961de259787f6e5: Status 404 returned error can't find the container with id 33f78a88170cedf84a5ba2b4a3b5c9ef0188aa57910f96e16961de259787f6e5
Nov 04 01:51:10.942349 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.942315    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:51:10.942424 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.942402    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:51:10.942499 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:10.942432    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:51:11.049016 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:11.048986    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67" event={"ID":"1bd672da-f401-43ae-a6bd-52247191c3a9","Type":"ContainerStarted","Data":"e532191a456a0328a30d1287555aaf9e87340cbeffdc0c6170aba21af656f3d4"}
Nov 04 01:51:11.049271 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:11.049034    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67" event={"ID":"1bd672da-f401-43ae-a6bd-52247191c3a9","Type":"ContainerStarted","Data":"33f78a88170cedf84a5ba2b4a3b5c9ef0188aa57910f96e16961de259787f6e5"}
Nov 04 01:51:11.170868 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:11.170810    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67" podStartSLOduration=1.170795522 podStartE2EDuration="1.170795522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:51:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:51:11.168686688 +0000 UTC m=+956.289964213" watchObservedRunningTime="2026-11-04 01:51:11.170795522 +0000 UTC m=+956.292073425"
Nov 04 01:51:11.511882 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:11.511848    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:11.998183 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:51:11.998146    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:51:12.513404 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:12.513370    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:51:12.513849 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:12.513427    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67" podUID="1bd672da-f401-43ae-a6bd-52247191c3a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 04 01:51:13.512710 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:13.512671    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:51:13.512901 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:13.512756    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67" podUID="1bd672da-f401-43ae-a6bd-52247191c3a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 04 01:51:14.512721 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:14.512686    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:51:14.513153 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:14.512753    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67" podUID="1bd672da-f401-43ae-a6bd-52247191c3a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 04 01:51:15.511843 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:15.511805    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:51:15.512068 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:15.511880    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67" podUID="1bd672da-f401-43ae-a6bd-52247191c3a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 04 01:51:16.512303 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:16.512268    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:51:16.512722 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:16.512339    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67" podUID="1bd672da-f401-43ae-a6bd-52247191c3a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 04 01:51:17.816471 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:17.816410    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:17.817218 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:17.817181    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:17.819661 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:17.819642    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:51:22.809145 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:22.809102    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"]
Nov 04 01:51:22.809682 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:22.809365    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9" podUID="829a880f-8e4d-40eb-ac51-309a55583ada" containerName="istio-proxy" containerID="cri-o://b79f3ca2d8e7da32b20225f9da43f9b78c117d5a05abe24979d5abf331a34591" gracePeriod=30
Nov 04 01:51:28.081387 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.081328    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:51:28.172576 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.172542    6116 generic.go:356] "Generic (PLEG): container finished" podID="829a880f-8e4d-40eb-ac51-309a55583ada" containerID="b79f3ca2d8e7da32b20225f9da43f9b78c117d5a05abe24979d5abf331a34591" exitCode=0
Nov 04 01:51:28.172760 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.172579    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9" event={"ID":"829a880f-8e4d-40eb-ac51-309a55583ada","Type":"ContainerDied","Data":"b79f3ca2d8e7da32b20225f9da43f9b78c117d5a05abe24979d5abf331a34591"}
Nov 04 01:51:28.172760 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.172612    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"
Nov 04 01:51:28.172760 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.172629    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9" event={"ID":"829a880f-8e4d-40eb-ac51-309a55583ada","Type":"ContainerDied","Data":"baf795926c3f739ceae7c99b50094b49f5464eb1afb800d41c4c35f5bf31b151"}
Nov 04 01:51:28.172760 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.172657    6116 scope.go:122] "RemoveContainer" containerID="b79f3ca2d8e7da32b20225f9da43f9b78c117d5a05abe24979d5abf331a34591"
Nov 04 01:51:28.182103 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.182082    6116 scope.go:122] "RemoveContainer" containerID="b79f3ca2d8e7da32b20225f9da43f9b78c117d5a05abe24979d5abf331a34591"
Nov 04 01:51:28.182422 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:51:28.182398    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b79f3ca2d8e7da32b20225f9da43f9b78c117d5a05abe24979d5abf331a34591\": container with ID starting with b79f3ca2d8e7da32b20225f9da43f9b78c117d5a05abe24979d5abf331a34591 not found: ID does not exist" containerID="b79f3ca2d8e7da32b20225f9da43f9b78c117d5a05abe24979d5abf331a34591"
Nov 04 01:51:28.182561 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.182438    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b79f3ca2d8e7da32b20225f9da43f9b78c117d5a05abe24979d5abf331a34591"} err="failed to get container status \"b79f3ca2d8e7da32b20225f9da43f9b78c117d5a05abe24979d5abf331a34591\": rpc error: code = NotFound desc = could not find container \"b79f3ca2d8e7da32b20225f9da43f9b78c117d5a05abe24979d5abf331a34591\": container with ID starting with b79f3ca2d8e7da32b20225f9da43f9b78c117d5a05abe24979d5abf331a34591 not found: ID does not exist"
Nov 04 01:51:28.233504 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.233464    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/829a880f-8e4d-40eb-ac51-309a55583ada-istio-token\" (UniqueName: \"kubernetes.io/projected/829a880f-8e4d-40eb-ac51-309a55583ada-istio-token\") pod \"829a880f-8e4d-40eb-ac51-309a55583ada\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") "
Nov 04 01:51:28.233695 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.233526    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/829a880f-8e4d-40eb-ac51-309a55583ada-kube-api-access-j2j7g\" (UniqueName: \"kubernetes.io/projected/829a880f-8e4d-40eb-ac51-309a55583ada-kube-api-access-j2j7g\") pod \"829a880f-8e4d-40eb-ac51-309a55583ada\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") "
Nov 04 01:51:28.233695 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.233550    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-istio-envoy\") pod \"829a880f-8e4d-40eb-ac51-309a55583ada\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") "
Nov 04 01:51:28.233695 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.233574    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/829a880f-8e4d-40eb-ac51-309a55583ada-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/829a880f-8e4d-40eb-ac51-309a55583ada-istiod-ca-cert\") pod \"829a880f-8e4d-40eb-ac51-309a55583ada\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") "
Nov 04 01:51:28.233695 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.233610    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-credential-socket\") pod \"829a880f-8e4d-40eb-ac51-309a55583ada\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") "
Nov 04 01:51:28.233695 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.233693    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-istio-data\") pod \"829a880f-8e4d-40eb-ac51-309a55583ada\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") "
Nov 04 01:51:28.233954 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.233837    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-workload-socket\") pod \"829a880f-8e4d-40eb-ac51-309a55583ada\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") "
Nov 04 01:51:28.233954 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.233918    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/829a880f-8e4d-40eb-ac51-309a55583ada-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/829a880f-8e4d-40eb-ac51-309a55583ada-istio-podinfo\") pod \"829a880f-8e4d-40eb-ac51-309a55583ada\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") "
Nov 04 01:51:28.234072 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.233976    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-workload-certs\") pod \"829a880f-8e4d-40eb-ac51-309a55583ada\" (UID: \"829a880f-8e4d-40eb-ac51-309a55583ada\") "
Nov 04 01:51:28.234072 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.233970    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-credential-socket" pod "829a880f-8e4d-40eb-ac51-309a55583ada" (UID: "829a880f-8e4d-40eb-ac51-309a55583ada"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:51:28.234072 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.233981    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-istio-data" pod "829a880f-8e4d-40eb-ac51-309a55583ada" (UID: "829a880f-8e4d-40eb-ac51-309a55583ada"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:51:28.234072 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.234028    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/829a880f-8e4d-40eb-ac51-309a55583ada-istiod-ca-cert" pod "829a880f-8e4d-40eb-ac51-309a55583ada" (UID: "829a880f-8e4d-40eb-ac51-309a55583ada"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:51:28.234420 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.234175    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-workload-socket" pod "829a880f-8e4d-40eb-ac51-309a55583ada" (UID: "829a880f-8e4d-40eb-ac51-309a55583ada"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:51:28.234420 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.234387    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-workload-certs" pod "829a880f-8e4d-40eb-ac51-309a55583ada" (UID: "829a880f-8e4d-40eb-ac51-309a55583ada"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:51:28.234612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.234596    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/829a880f-8e4d-40eb-ac51-309a55583ada-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:51:28.234670 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.234621    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:51:28.234781 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.234712    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:51:28.234837 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.234807    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:51:28.234837 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.234820    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:51:28.235983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.235959    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-istio-envoy" pod "829a880f-8e4d-40eb-ac51-309a55583ada" (UID: "829a880f-8e4d-40eb-ac51-309a55583ada"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:51:28.236105 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.236006    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/829a880f-8e4d-40eb-ac51-309a55583ada-kube-api-access-j2j7g" pod "829a880f-8e4d-40eb-ac51-309a55583ada" (UID: "829a880f-8e4d-40eb-ac51-309a55583ada"). InnerVolumeSpecName "kube-api-access-j2j7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:51:28.236105 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.236031    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/829a880f-8e4d-40eb-ac51-309a55583ada-istio-token" pod "829a880f-8e4d-40eb-ac51-309a55583ada" (UID: "829a880f-8e4d-40eb-ac51-309a55583ada"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:51:28.236185 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.236150    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/829a880f-8e4d-40eb-ac51-309a55583ada-istio-podinfo" pod "829a880f-8e4d-40eb-ac51-309a55583ada" (UID: "829a880f-8e4d-40eb-ac51-309a55583ada"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:51:28.335990 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.335905    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/829a880f-8e4d-40eb-ac51-309a55583ada-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:51:28.335990 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.335937    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/829a880f-8e4d-40eb-ac51-309a55583ada-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:51:28.335990 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.335946    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2j7g\" (UniqueName: \"kubernetes.io/projected/829a880f-8e4d-40eb-ac51-309a55583ada-kube-api-access-j2j7g\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:51:28.335990 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.335956    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/829a880f-8e4d-40eb-ac51-309a55583ada-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:51:28.600557 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.600472    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"]
Nov 04 01:51:28.603857 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:28.603835    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9"]
Nov 04 01:51:29.028684 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:29.028642    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="829a880f-8e4d-40eb-ac51-309a55583ada" path="/var/lib/kubelet/pods/829a880f-8e4d-40eb-ac51-309a55583ada/volumes"
Nov 04 01:51:37.555347 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:37.555311    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:51:41.097340 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.097303    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"]
Nov 04 01:51:41.097843 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.097830    6116 cpu_manager.go:414] "containerMap: removing container" podUID="829a880f-8e4d-40eb-ac51-309a55583ada" containerName="istio-proxy"
Nov 04 01:51:41.097906 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.097844    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="829a880f-8e4d-40eb-ac51-309a55583ada" containerName="istio-proxy"
Nov 04 01:51:41.097940 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.097928    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="829a880f-8e4d-40eb-ac51-309a55583ada" containerName="istio-proxy"
Nov 04 01:51:41.102555 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.102536    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.192867 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.192821    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"]
Nov 04 01:51:41.202906 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.202871    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-credential-socket\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.202906 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.202910    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-token\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.203120 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.203058    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-workload-socket\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.203120 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.203091    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-podinfo\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.203209 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.203125    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istiod-ca-cert\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.203864 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.203277    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-envoy\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.204222 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.204168    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnckc\" (UniqueName: \"kubernetes.io/projected/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-kube-api-access-jnckc\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.204496 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.204468    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-workload-certs\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.206421 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.204664    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-data\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.306248 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.306210    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istiod-ca-cert\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.306604 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.306262    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-envoy\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.306604 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.306307    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnckc\" (UniqueName: \"kubernetes.io/projected/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-kube-api-access-jnckc\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.306604 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.306341    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-workload-certs\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.306604 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.306357    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-data\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.306604 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.306378    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-credential-socket\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.306604 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.306392    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-token\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.306604 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.306482    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-workload-socket\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.306604 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.306516    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-podinfo\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.307045 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.306981    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istiod-ca-cert\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.307347 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.307319    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-credential-socket\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.307559 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.307349    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-workload-certs\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.307559 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.307400    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-workload-socket\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.307736 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.307485    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-data\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.309406 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.309382    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-envoy\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.309549 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.309530    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-podinfo\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.374712 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.374634    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-token\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.374894 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.374772    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnckc\" (UniqueName: \"kubernetes.io/projected/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-kube-api-access-jnckc\") pod \"e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") " pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.415097 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.415061    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:41.874072 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.874039    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"]
Nov 04 01:51:41.875604 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:51:41.875562    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70fa17e3_fb7b_4ac0_b7ba_ae560f39f054.slice/crio-b9d0961b097e6df1a98bb417d6dcf44c0f77061c409c6b2111baae9dcae5e02b WatchSource:0}: Error finding container b9d0961b097e6df1a98bb417d6dcf44c0f77061c409c6b2111baae9dcae5e02b: Status 404 returned error can't find the container with id b9d0961b097e6df1a98bb417d6dcf44c0f77061c409c6b2111baae9dcae5e02b
Nov 04 01:51:41.878109 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.878073    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:51:41.878198 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.878163    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:51:41.878198 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:41.878194    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:51:42.005273 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:51:42.005238    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:51:42.283439 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:42.283390    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh" event={"ID":"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054","Type":"ContainerStarted","Data":"bd15940d0c0d46fd7a6b14de033e5563ad8069a3802aabbf7fd164d9b0f4654c"}
Nov 04 01:51:42.283923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:42.283490    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh" event={"ID":"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054","Type":"ContainerStarted","Data":"b9d0961b097e6df1a98bb417d6dcf44c0f77061c409c6b2111baae9dcae5e02b"}
Nov 04 01:51:42.407469 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:42.407408    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh" podStartSLOduration=1.407393616 podStartE2EDuration="1.407393616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:51:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:51:42.403998811 +0000 UTC m=+987.525276330" watchObservedRunningTime="2026-11-04 01:51:42.407393616 +0000 UTC m=+987.528671140"
Nov 04 01:51:42.416339 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:42.416314    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:42.422461 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:42.422401    6116 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:51:42.422662 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:42.422493    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh" podUID="70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 01:51:43.416139 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:43.416107    6116 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:51:43.416605 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:43.416167    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh" podUID="70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 01:51:44.415688 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:44.415654    6116 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:51:44.415870 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:44.415704    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh" podUID="70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 01:51:45.415493 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:45.415429    6116 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:51:45.415916 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:45.415543    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh" podUID="70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 01:51:45.931640 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:45.931600    6116 patch_prober.go:29] interesting pod/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:51:45.931813 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:45.931684    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6" podUID="35deac99-14d0-416d-9c8d-c8abb03f5f7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:51:46.416362 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:46.416332    6116 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:51:46.416902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:46.416391    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh" podUID="70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 01:51:47.415432 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:47.415395    6116 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:51:47.415616 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:47.415474    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh" podUID="70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 01:51:48.333600 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.333482    6116 generic.go:356] "Generic (PLEG): container finished" podID="35deac99-14d0-416d-9c8d-c8abb03f5f7d" containerID="ef2cd128dacca8851fb28b42841f1f650dfc53652a7704966971dae8a36b09de" exitCode=0
Nov 04 01:51:48.333600 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.333550    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6" event={"ID":"35deac99-14d0-416d-9c8d-c8abb03f5f7d","Type":"ContainerDied","Data":"ef2cd128dacca8851fb28b42841f1f650dfc53652a7704966971dae8a36b09de"}
Nov 04 01:51:48.334193 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.333969    6116 scope.go:122] "RemoveContainer" containerID="ef2cd128dacca8851fb28b42841f1f650dfc53652a7704966971dae8a36b09de"
Nov 04 01:51:48.335520 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.335485    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:51:48.335636 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.335575    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:51:48.335636 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.335623    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:51:48.416110 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.416083    6116 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:51:48.416252 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.416132    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh" podUID="70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 01:51:48.840580 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.840537    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"]
Nov 04 01:51:48.844430 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.844402    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"
Nov 04 01:51:48.899378 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.899344    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Nov 04 01:51:48.899668 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.899364    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Nov 04 01:51:48.899777 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.899373    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Nov 04 01:51:48.899842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.899541    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-5kbxt\""
Nov 04 01:51:48.899842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.899598    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-84cffd7d1248\""
Nov 04 01:51:48.899842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.899684    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Nov 04 01:51:48.903066 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.903040    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"]
Nov 04 01:51:48.928724 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.928688    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-plugins-config-volume\") pod \"payload-processing-external-model-586b974f6-dt8ql\" (UID: \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\") " pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"
Nov 04 01:51:48.928922 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.928856    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-provider-credentials\") pod \"payload-processing-external-model-586b974f6-dt8ql\" (UID: \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\") " pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"
Nov 04 01:51:48.928922 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.928899    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djl26\" (UniqueName: \"kubernetes.io/projected/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-kube-api-access-djl26\") pod \"payload-processing-external-model-586b974f6-dt8ql\" (UID: \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\") " pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"
Nov 04 01:51:48.929108 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:48.929081    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-routing-overlay-volume\") pod \"payload-processing-external-model-586b974f6-dt8ql\" (UID: \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\") " pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"
Nov 04 01:51:49.034082 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:49.034045    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-plugins-config-volume\") pod \"payload-processing-external-model-586b974f6-dt8ql\" (UID: \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\") " pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"
Nov 04 01:51:49.034483 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:49.034418    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-provider-credentials\") pod \"payload-processing-external-model-586b974f6-dt8ql\" (UID: \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\") " pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"
Nov 04 01:51:49.034649 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:49.034630    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djl26\" (UniqueName: \"kubernetes.io/projected/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-kube-api-access-djl26\") pod \"payload-processing-external-model-586b974f6-dt8ql\" (UID: \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\") " pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"
Nov 04 01:51:49.034879 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:49.034860    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-routing-overlay-volume\") pod \"payload-processing-external-model-586b974f6-dt8ql\" (UID: \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\") " pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"
Nov 04 01:51:49.035054 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:49.035034    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-plugins-config-volume\") pod \"payload-processing-external-model-586b974f6-dt8ql\" (UID: \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\") " pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"
Nov 04 01:51:49.035697 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:49.035671    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-routing-overlay-volume\") pod \"payload-processing-external-model-586b974f6-dt8ql\" (UID: \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\") " pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"
Nov 04 01:51:49.041287 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:49.040406    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-provider-credentials\") pod \"payload-processing-external-model-586b974f6-dt8ql\" (UID: \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\") " pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"
Nov 04 01:51:49.096934 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:49.096849    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djl26\" (UniqueName: \"kubernetes.io/projected/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-kube-api-access-djl26\") pod \"payload-processing-external-model-586b974f6-dt8ql\" (UID: \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\") " pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"
Nov 04 01:51:49.160975 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:49.160674    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"
Nov 04 01:51:49.362682 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:49.361562    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6" event={"ID":"35deac99-14d0-416d-9c8d-c8abb03f5f7d","Type":"ContainerStarted","Data":"6421160f46b28a73eef60df1aa736b855a28f0bc93c3199f948cc69ba3514c36"}
Nov 04 01:51:49.496236 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:49.496193    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:49.496576 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:49.496556    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:49.497653 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:49.497627    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:51:49.601152 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:49.600528    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"]
Nov 04 01:51:49.604264 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:51:49.604228    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5986cf86_3ff4_4a14_aaa3_38e5aaedf59d.slice/crio-923a24d38bb962060c7b614e8219a87863730e2459f14fcfe9fe4352d88d1c01 WatchSource:0}: Error finding container 923a24d38bb962060c7b614e8219a87863730e2459f14fcfe9fe4352d88d1c01: Status 404 returned error can't find the container with id 923a24d38bb962060c7b614e8219a87863730e2459f14fcfe9fe4352d88d1c01
Nov 04 01:51:49.761473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:49.750801    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:51:49.761473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:49.750918    6116 patch_prober.go:29] interesting pod/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 04 01:51:49.761473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:49.750971    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6" podUID="35deac99-14d0-416d-9c8d-c8abb03f5f7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 04 01:51:50.271418 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.271352    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"]
Nov 04 01:51:50.276310 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.276289    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"
Nov 04 01:51:50.311417 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.311386    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"]
Nov 04 01:51:50.328443 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.328415    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-754dba0eabb0\""
Nov 04 01:51:50.377519 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.377482    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql" event={"ID":"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d","Type":"ContainerStarted","Data":"dbf9c3d2784b0351bfdc010875e7c88902f14c6785e13d0652b0c0cb9dd67710"}
Nov 04 01:51:50.377988 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.377555    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"
Nov 04 01:51:50.377988 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.377589    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql" event={"ID":"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d","Type":"ContainerStarted","Data":"923a24d38bb962060c7b614e8219a87863730e2459f14fcfe9fe4352d88d1c01"}
Nov 04 01:51:50.386742 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.386709    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/233cb7a1-2c92-4544-afe5-5aae953288e1-routing-overlay-volume\") pod \"payload-processing-external-model-5497dccdcc-bbs4k\" (UID: \"233cb7a1-2c92-4544-afe5-5aae953288e1\") " pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"
Nov 04 01:51:50.386905 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.386888    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials\") pod \"payload-processing-external-model-5497dccdcc-bbs4k\" (UID: \"233cb7a1-2c92-4544-afe5-5aae953288e1\") " pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"
Nov 04 01:51:50.386981 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.386943    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c56hd\" (UniqueName: \"kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-kube-api-access-c56hd\") pod \"payload-processing-external-model-5497dccdcc-bbs4k\" (UID: \"233cb7a1-2c92-4544-afe5-5aae953288e1\") " pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"
Nov 04 01:51:50.387022 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.387007    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/233cb7a1-2c92-4544-afe5-5aae953288e1-plugins-config-volume\") pod \"payload-processing-external-model-5497dccdcc-bbs4k\" (UID: \"233cb7a1-2c92-4544-afe5-5aae953288e1\") " pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"
Nov 04 01:51:50.488575 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.488533    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c56hd\" (UniqueName: \"kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-kube-api-access-c56hd\") pod \"payload-processing-external-model-5497dccdcc-bbs4k\" (UID: \"233cb7a1-2c92-4544-afe5-5aae953288e1\") " pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"
Nov 04 01:51:50.488575 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.488586    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/233cb7a1-2c92-4544-afe5-5aae953288e1-plugins-config-volume\") pod \"payload-processing-external-model-5497dccdcc-bbs4k\" (UID: \"233cb7a1-2c92-4544-afe5-5aae953288e1\") " pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"
Nov 04 01:51:50.488902 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.488867    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/233cb7a1-2c92-4544-afe5-5aae953288e1-routing-overlay-volume\") pod \"payload-processing-external-model-5497dccdcc-bbs4k\" (UID: \"233cb7a1-2c92-4544-afe5-5aae953288e1\") " pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"
Nov 04 01:51:50.489309 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.489205    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials\") pod \"payload-processing-external-model-5497dccdcc-bbs4k\" (UID: \"233cb7a1-2c92-4544-afe5-5aae953288e1\") " pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"
Nov 04 01:51:50.489583 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.489565    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/233cb7a1-2c92-4544-afe5-5aae953288e1-plugins-config-volume\") pod \"payload-processing-external-model-5497dccdcc-bbs4k\" (UID: \"233cb7a1-2c92-4544-afe5-5aae953288e1\") " pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"
Nov 04 01:51:50.489701 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.489597    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/233cb7a1-2c92-4544-afe5-5aae953288e1-routing-overlay-volume\") pod \"payload-processing-external-model-5497dccdcc-bbs4k\" (UID: \"233cb7a1-2c92-4544-afe5-5aae953288e1\") " pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"
Nov 04 01:51:50.491865 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.491836    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials\") pod \"payload-processing-external-model-5497dccdcc-bbs4k\" (UID: \"233cb7a1-2c92-4544-afe5-5aae953288e1\") " pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"
Nov 04 01:51:50.572117 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.572035    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c56hd\" (UniqueName: \"kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-kube-api-access-c56hd\") pod \"payload-processing-external-model-5497dccdcc-bbs4k\" (UID: \"233cb7a1-2c92-4544-afe5-5aae953288e1\") " pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"
Nov 04 01:51:50.589524 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.589496    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"
Nov 04 01:51:50.770496 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.753807    6116 patch_prober.go:29] interesting pod/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 04 01:51:50.770496 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.753865    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6" podUID="35deac99-14d0-416d-9c8d-c8abb03f5f7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 04 01:51:50.973925 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.972202    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql" podStartSLOduration=2.972182364 podStartE2EDuration="2.972182364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:51:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:51:50.569278732 +0000 UTC m=+995.690556258" watchObservedRunningTime="2026-11-04 01:51:50.972182364 +0000 UTC m=+996.093459891"
Nov 04 01:51:50.975307 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:50.975280    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"]
Nov 04 01:51:50.976046 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:51:50.976020    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod233cb7a1_2c92_4544_afe5_5aae953288e1.slice/crio-b3746d9fade88be3205242fb4d5b3bbdd2a3f7a10c9063f654dce3f05f9a2c9b WatchSource:0}: Error finding container b3746d9fade88be3205242fb4d5b3bbdd2a3f7a10c9063f654dce3f05f9a2c9b: Status 404 returned error can't find the container with id b3746d9fade88be3205242fb4d5b3bbdd2a3f7a10c9063f654dce3f05f9a2c9b
Nov 04 01:51:51.385415 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:51.385316    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k" event={"ID":"233cb7a1-2c92-4544-afe5-5aae953288e1","Type":"ContainerStarted","Data":"cc683b087ce900646610d30fc54e45c04ca554222c2446f27273a4f5b3fc93f9"}
Nov 04 01:51:51.385415 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:51.385377    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k" event={"ID":"233cb7a1-2c92-4544-afe5-5aae953288e1","Type":"ContainerStarted","Data":"b3746d9fade88be3205242fb4d5b3bbdd2a3f7a10c9063f654dce3f05f9a2c9b"}
Nov 04 01:51:51.385911 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:51.385608    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"
Nov 04 01:51:51.503247 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:51.503187    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k" podStartSLOduration=1.5031642729999999 podStartE2EDuration="1.503164273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:51:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:51:51.500685839 +0000 UTC m=+996.621963365" watchObservedRunningTime="2026-11-04 01:51:51.503164273 +0000 UTC m=+996.624441800"
Nov 04 01:51:51.752805 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:51.752769    6116 patch_prober.go:29] interesting pod/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 04 01:51:51.753098 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:51.753066    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6" podUID="35deac99-14d0-416d-9c8d-c8abb03f5f7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 04 01:51:52.747849 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:52.747812    6116 patch_prober.go:29] interesting pod/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 04 01:51:52.748401 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:52.748368    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6" podUID="35deac99-14d0-416d-9c8d-c8abb03f5f7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 04 01:51:53.753533 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:53.751827    6116 patch_prober.go:29] interesting pod/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 04 01:51:53.753533 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:53.751902    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6" podUID="35deac99-14d0-416d-9c8d-c8abb03f5f7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 04 01:51:54.601868 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.601830    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"]
Nov 04 01:51:54.606637 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.606611    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.708997 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.708967    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-82706f\"/\"distinct-2-w2-82706f-kserve-self-signed-certs\""
Nov 04 01:51:54.711719 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.711697    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"]
Nov 04 01:51:54.740811 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.740778    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-tmp-dir\") pod \"distinct-2-w2-82706f-kserve-7db8d86777-4wjbn\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.741214 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.740839    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2krlb\" (UniqueName: \"kubernetes.io/projected/105b1344-7994-4f1b-b6c6-1476c606d52f-kube-api-access-2krlb\") pod \"distinct-2-w2-82706f-kserve-7db8d86777-4wjbn\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.741214 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.741017    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-dshm\") pod \"distinct-2-w2-82706f-kserve-7db8d86777-4wjbn\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.741214 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.741117    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-model-cache\") pod \"distinct-2-w2-82706f-kserve-7db8d86777-4wjbn\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.741214 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.741174    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-home\") pod \"distinct-2-w2-82706f-kserve-7db8d86777-4wjbn\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.741565 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.741317    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/105b1344-7994-4f1b-b6c6-1476c606d52f-tls-certs\") pod \"distinct-2-w2-82706f-kserve-7db8d86777-4wjbn\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.757789 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.757758    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:51:54.760600 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.759226    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:51:54.760600 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.759802    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:51:54.842298 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.842264    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/105b1344-7994-4f1b-b6c6-1476c606d52f-tls-certs\") pod \"distinct-2-w2-82706f-kserve-7db8d86777-4wjbn\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.842298 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.842300    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-tmp-dir\") pod \"distinct-2-w2-82706f-kserve-7db8d86777-4wjbn\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.842601 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.842330    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2krlb\" (UniqueName: \"kubernetes.io/projected/105b1344-7994-4f1b-b6c6-1476c606d52f-kube-api-access-2krlb\") pod \"distinct-2-w2-82706f-kserve-7db8d86777-4wjbn\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.842601 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.842400    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-dshm\") pod \"distinct-2-w2-82706f-kserve-7db8d86777-4wjbn\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.842601 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.842489    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-model-cache\") pod \"distinct-2-w2-82706f-kserve-7db8d86777-4wjbn\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.842601 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.842539    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-home\") pod \"distinct-2-w2-82706f-kserve-7db8d86777-4wjbn\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.843085 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.843058    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-tmp-dir\") pod \"distinct-2-w2-82706f-kserve-7db8d86777-4wjbn\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.843085 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.843081    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-home\") pod \"distinct-2-w2-82706f-kserve-7db8d86777-4wjbn\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.843226 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.843100    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-model-cache\") pod \"distinct-2-w2-82706f-kserve-7db8d86777-4wjbn\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.845281 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.845260    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-dshm\") pod \"distinct-2-w2-82706f-kserve-7db8d86777-4wjbn\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.845649 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.845627    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/105b1344-7994-4f1b-b6c6-1476c606d52f-tls-certs\") pod \"distinct-2-w2-82706f-kserve-7db8d86777-4wjbn\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.904145 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.904111    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2krlb\" (UniqueName: \"kubernetes.io/projected/105b1344-7994-4f1b-b6c6-1476c606d52f-kube-api-access-2krlb\") pod \"distinct-2-w2-82706f-kserve-7db8d86777-4wjbn\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") " pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:54.919918 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:54.919890    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:55.323691 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:55.323670    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"]
Nov 04 01:51:55.324872 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:51:55.324844    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod105b1344_7994_4f1b_b6c6_1476c606d52f.slice/crio-28376d1861f75f0a146615978a317aa0a46ae9e1e9ecc42817615f14d34fcf31 WatchSource:0}: Error finding container 28376d1861f75f0a146615978a317aa0a46ae9e1e9ecc42817615f14d34fcf31: Status 404 returned error can't find the container with id 28376d1861f75f0a146615978a317aa0a46ae9e1e9ecc42817615f14d34fcf31
Nov 04 01:51:55.428565 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:55.428482    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn" event={"ID":"105b1344-7994-4f1b-b6c6-1476c606d52f","Type":"ContainerStarted","Data":"c64111a1ce9bd38457d732493d29f1ea44dab5fea28fa9398ef962857da6ff84"}
Nov 04 01:51:55.428565 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:55.428549    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn" event={"ID":"105b1344-7994-4f1b-b6c6-1476c606d52f","Type":"ContainerStarted","Data":"28376d1861f75f0a146615978a317aa0a46ae9e1e9ecc42817615f14d34fcf31"}
Nov 04 01:51:55.546664 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:55.546610    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn" podStartSLOduration=1.546595178 podStartE2EDuration="1.546595178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:51:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:51:55.544303568 +0000 UTC m=+1000.665581095" watchObservedRunningTime="2026-11-04 01:51:55.546595178 +0000 UTC m=+1000.667872703"
Nov 04 01:51:55.920551 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:55.920519    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:51:58.342864 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.342831    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"]
Nov 04 01:51:58.347394 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.347374    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.400149 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.400119    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-7af38e-openshift-default-dockercfg-pbl8c\""
Nov 04 01:51:58.452958 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.452927    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"]
Nov 04 01:51:58.488380 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.488346    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-data\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.488554 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.488393    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-podinfo\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.488554 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.488423    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-workload-socket\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.488554 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.488522    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zpd7\" (UniqueName: \"kubernetes.io/projected/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-kube-api-access-2zpd7\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.488656 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.488612    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-envoy\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.488693 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.488659    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istiod-ca-cert\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.488765 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.488752    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-credential-socket\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.488801 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.488781    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-token\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.488841 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.488800    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-workload-certs\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.590125 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.590089    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-credential-socket\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.590343 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.590145    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-token\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.590343 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.590173    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-workload-certs\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.590343 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.590228    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-data\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.590343 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.590264    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-podinfo\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.590343 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.590290    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-workload-socket\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.590627 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.590346    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zpd7\" (UniqueName: \"kubernetes.io/projected/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-kube-api-access-2zpd7\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.590627 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.590406    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-envoy\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.590627 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.590516    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-credential-socket\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.590627 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.590551    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istiod-ca-cert\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.590819 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.590726    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-workload-certs\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.590819 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.590789    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-workload-socket\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.590929 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.590896    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-data\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.591278 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.591245    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istiod-ca-cert\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.592779 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.592754    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-envoy\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.592906 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.592858    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-podinfo\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.651311 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.651282    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-token\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.651476 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.651359    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zpd7\" (UniqueName: \"kubernetes.io/projected/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-kube-api-access-2zpd7\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:58.661296 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:58.661276    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:51:59.088518 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:51:59.088490    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89a13dcf_c8a3_4092_8ff0_d63fb3ae874f.slice/crio-86818ead554be7d7346e46b13c280c9e0a060169a07ff5237b11612730745d82 WatchSource:0}: Error finding container 86818ead554be7d7346e46b13c280c9e0a060169a07ff5237b11612730745d82: Status 404 returned error can't find the container with id 86818ead554be7d7346e46b13c280c9e0a060169a07ff5237b11612730745d82
Nov 04 01:51:59.091474 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:59.091409    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:51:59.091579 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:59.091496    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:51:59.091579 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:59.091526    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:51:59.093775 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:59.093750    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"]
Nov 04 01:51:59.459036 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:59.458999    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh" event={"ID":"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f","Type":"ContainerStarted","Data":"36de27cba574e95cee6d78ef560b41db4885372d5be411ca1a530ec057a52193"}
Nov 04 01:51:59.459425 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:59.459058    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh" event={"ID":"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f","Type":"ContainerStarted","Data":"86818ead554be7d7346e46b13c280c9e0a060169a07ff5237b11612730745d82"}
Nov 04 01:51:59.624629 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:59.624576    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh" podStartSLOduration=1.624561778 podStartE2EDuration="1.624561778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:51:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:51:59.621532617 +0000 UTC m=+1004.742810143" watchObservedRunningTime="2026-11-04 01:51:59.624561778 +0000 UTC m=+1004.745839304"
Nov 04 01:51:59.661760 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:51:59.661722    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:52:00.665787 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:00.665751    6116 patch_prober.go:29] interesting pod/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:52:00.666374 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:00.666343    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh" podUID="89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:52:01.389245 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:01.389218    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"
Nov 04 01:52:01.664951 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:01.664920    6116 patch_prober.go:29] interesting pod/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:52:01.665551 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:01.665519    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh" podUID="89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:52:02.403268 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:02.403235    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"
Nov 04 01:52:02.579813 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:02.579779    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"]
Nov 04 01:52:02.668855 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:02.668765    6116 patch_prober.go:29] interesting pod/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:52:02.669114 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:02.669084    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh" podUID="89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:52:02.774252 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:02.774215    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:52:03.661921 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:03.661883    6116 patch_prober.go:29] interesting pod/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:52:03.662574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:03.662532    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh" podUID="89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:52:04.662053 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:04.662013    6116 patch_prober.go:29] interesting pod/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:52:04.662674 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:04.662625    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh" podUID="89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:52:04.920640 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:04.920553    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:52:04.933066 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:04.933039    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:52:05.530911 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:05.530880    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:52:05.662497 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:05.662439    6116 patch_prober.go:29] interesting pod/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:52:05.662905 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:05.662514    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh" podUID="89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:52:06.667228 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:06.667194    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:52:06.667700 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:06.667632    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:52:06.668324 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:06.668308    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:52:07.580411 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:07.580360    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql" podUID="5986cf86-3ff4-4a14-aaa3-38e5aaedf59d" containerName="payload-processing" containerID="cri-o://dbf9c3d2784b0351bfdc010875e7c88902f14c6785e13d0652b0c0cb9dd67710" gracePeriod=25
Nov 04 01:52:07.835395 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:07.835329    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"
Nov 04 01:52:07.958350 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:07.958322    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-routing-overlay-volume\") pod \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\" (UID: \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\") "
Nov 04 01:52:07.958594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:07.958505    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-plugins-config-volume\") pod \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\" (UID: \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\") "
Nov 04 01:52:07.958594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:07.958542    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-provider-credentials\" (UniqueName: \"kubernetes.io/projected/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-provider-credentials\") pod \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\" (UID: \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\") "
Nov 04 01:52:07.958718 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:07.958647    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-kube-api-access-djl26\" (UniqueName: \"kubernetes.io/projected/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-kube-api-access-djl26\") pod \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\" (UID: \"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d\") "
Nov 04 01:52:07.958802 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:07.958763    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-routing-overlay-volume" pod "5986cf86-3ff4-4a14-aaa3-38e5aaedf59d" (UID: "5986cf86-3ff4-4a14-aaa3-38e5aaedf59d"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:52:07.958936 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:07.958885    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-plugins-config-volume" pod "5986cf86-3ff4-4a14-aaa3-38e5aaedf59d" (UID: "5986cf86-3ff4-4a14-aaa3-38e5aaedf59d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:52:07.959084 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:07.959067    6116 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-routing-overlay-volume\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:07.959132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:07.959090    6116 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-plugins-config-volume\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:07.961119 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:07.961084    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-kube-api-access-djl26" pod "5986cf86-3ff4-4a14-aaa3-38e5aaedf59d" (UID: "5986cf86-3ff4-4a14-aaa3-38e5aaedf59d"). InnerVolumeSpecName "kube-api-access-djl26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:52:07.961236 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:07.961126    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-provider-credentials" pod "5986cf86-3ff4-4a14-aaa3-38e5aaedf59d" (UID: "5986cf86-3ff4-4a14-aaa3-38e5aaedf59d"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:52:08.060186 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:08.060154    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djl26\" (UniqueName: \"kubernetes.io/projected/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-kube-api-access-djl26\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:08.060186 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:08.060186    6116 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d-provider-credentials\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:08.538400 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:08.538369    6116 generic.go:356] "Generic (PLEG): container finished" podID="5986cf86-3ff4-4a14-aaa3-38e5aaedf59d" containerID="dbf9c3d2784b0351bfdc010875e7c88902f14c6785e13d0652b0c0cb9dd67710" exitCode=0
Nov 04 01:52:08.538623 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:08.538417    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"
Nov 04 01:52:08.538623 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:08.538467    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql" event={"ID":"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d","Type":"ContainerDied","Data":"dbf9c3d2784b0351bfdc010875e7c88902f14c6785e13d0652b0c0cb9dd67710"}
Nov 04 01:52:08.538623 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:08.538532    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-586b974f6-dt8ql" event={"ID":"5986cf86-3ff4-4a14-aaa3-38e5aaedf59d","Type":"ContainerDied","Data":"923a24d38bb962060c7b614e8219a87863730e2459f14fcfe9fe4352d88d1c01"}
Nov 04 01:52:08.538623 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:08.538560    6116 scope.go:122] "RemoveContainer" containerID="dbf9c3d2784b0351bfdc010875e7c88902f14c6785e13d0652b0c0cb9dd67710"
Nov 04 01:52:08.550776 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:08.550715    6116 scope.go:122] "RemoveContainer" containerID="dbf9c3d2784b0351bfdc010875e7c88902f14c6785e13d0652b0c0cb9dd67710"
Nov 04 01:52:08.551100 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:52:08.551067    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbf9c3d2784b0351bfdc010875e7c88902f14c6785e13d0652b0c0cb9dd67710\": container with ID starting with dbf9c3d2784b0351bfdc010875e7c88902f14c6785e13d0652b0c0cb9dd67710 not found: ID does not exist" containerID="dbf9c3d2784b0351bfdc010875e7c88902f14c6785e13d0652b0c0cb9dd67710"
Nov 04 01:52:08.551214 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:08.551110    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbf9c3d2784b0351bfdc010875e7c88902f14c6785e13d0652b0c0cb9dd67710"} err="failed to get container status \"dbf9c3d2784b0351bfdc010875e7c88902f14c6785e13d0652b0c0cb9dd67710\": rpc error: code = NotFound desc = could not find container \"dbf9c3d2784b0351bfdc010875e7c88902f14c6785e13d0652b0c0cb9dd67710\": container with ID starting with dbf9c3d2784b0351bfdc010875e7c88902f14c6785e13d0652b0c0cb9dd67710 not found: ID does not exist"
Nov 04 01:52:08.703768 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:08.703733    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"]
Nov 04 01:52:08.753074 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:08.753041    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-586b974f6-dt8ql"]
Nov 04 01:52:09.028662 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:09.028632    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5986cf86-3ff4-4a14-aaa3-38e5aaedf59d" path="/var/lib/kubelet/pods/5986cf86-3ff4-4a14-aaa3-38e5aaedf59d/volumes"
Nov 04 01:52:11.992003 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:52:11.991968    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:52:16.949310 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:16.949274    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk"]
Nov 04 01:52:16.949968 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:16.949948    6116 cpu_manager.go:414] "containerMap: removing container" podUID="5986cf86-3ff4-4a14-aaa3-38e5aaedf59d" containerName="payload-processing"
Nov 04 01:52:16.950083 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:16.949972    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5986cf86-3ff4-4a14-aaa3-38e5aaedf59d" containerName="payload-processing"
Nov 04 01:52:16.950137 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:16.950089    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="5986cf86-3ff4-4a14-aaa3-38e5aaedf59d" containerName="payload-processing"
Nov 04 01:52:16.955182 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:16.955161    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk"
Nov 04 01:52:17.011259 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:17.011223    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk"]
Nov 04 01:52:17.056683 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:17.056636    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 01:52:17.056683 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:17.056666    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:52:17.056683 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:17.056683    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bj8b6\""
Nov 04 01:52:17.057532 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:17.057517    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:52:17.059425 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:17.059403    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtgpc\" (UniqueName: \"kubernetes.io/projected/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-kube-api-access-dtgpc\") pod \"maas-api-revoke-sub-34d92457-tcskk\" (UID: \"d949886a-cd12-48ed-9d9a-6fb9e6df4fb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk"
Nov 04 01:52:17.059566 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:17.059552    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-34d92457-tcskk\" (UID: \"d949886a-cd12-48ed-9d9a-6fb9e6df4fb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk"
Nov 04 01:52:17.059618 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:17.059607    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-34d92457-tcskk\" (UID: \"d949886a-cd12-48ed-9d9a-6fb9e6df4fb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk"
Nov 04 01:52:17.160369 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:17.160336    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-34d92457-tcskk\" (UID: \"d949886a-cd12-48ed-9d9a-6fb9e6df4fb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk"
Nov 04 01:52:17.160566 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:17.160438    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtgpc\" (UniqueName: \"kubernetes.io/projected/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-kube-api-access-dtgpc\") pod \"maas-api-revoke-sub-34d92457-tcskk\" (UID: \"d949886a-cd12-48ed-9d9a-6fb9e6df4fb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk"
Nov 04 01:52:17.160623 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:17.160597    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-34d92457-tcskk\" (UID: \"d949886a-cd12-48ed-9d9a-6fb9e6df4fb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk"
Nov 04 01:52:17.161180 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:17.161154    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-34d92457-tcskk\" (UID: \"d949886a-cd12-48ed-9d9a-6fb9e6df4fb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk"
Nov 04 01:52:17.162829 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:17.162808    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-34d92457-tcskk\" (UID: \"d949886a-cd12-48ed-9d9a-6fb9e6df4fb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk"
Nov 04 01:52:17.221128 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:17.221046    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtgpc\" (UniqueName: \"kubernetes.io/projected/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-kube-api-access-dtgpc\") pod \"maas-api-revoke-sub-34d92457-tcskk\" (UID: \"d949886a-cd12-48ed-9d9a-6fb9e6df4fb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk"
Nov 04 01:52:17.268622 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:17.268591    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk"
Nov 04 01:52:17.644585 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:17.644559    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk"]
Nov 04 01:52:17.647042 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:52:17.647017    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd949886a_cd12_48ed_9d9a_6fb9e6df4fb8.slice/crio-d95b59ecd014aa304b7cdd9d56f0ff09fa7094d8226788e78a85e797c58bb62e WatchSource:0}: Error finding container d95b59ecd014aa304b7cdd9d56f0ff09fa7094d8226788e78a85e797c58bb62e: Status 404 returned error can't find the container with id d95b59ecd014aa304b7cdd9d56f0ff09fa7094d8226788e78a85e797c58bb62e
Nov 04 01:52:18.641060 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:18.641026    6116 generic.go:356] "Generic (PLEG): container finished" podID="d949886a-cd12-48ed-9d9a-6fb9e6df4fb8" containerID="40dc710bc642789a8f5c2d9d8898bf80652eb822b7c7325539ceb0bbd710a717" exitCode=0
Nov 04 01:52:18.641430 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:18.641066    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk" event={"ID":"d949886a-cd12-48ed-9d9a-6fb9e6df4fb8","Type":"ContainerDied","Data":"40dc710bc642789a8f5c2d9d8898bf80652eb822b7c7325539ceb0bbd710a717"}
Nov 04 01:52:18.641430 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:18.641132    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk" event={"ID":"d949886a-cd12-48ed-9d9a-6fb9e6df4fb8","Type":"ContainerStarted","Data":"d95b59ecd014aa304b7cdd9d56f0ff09fa7094d8226788e78a85e797c58bb62e"}
Nov 04 01:52:19.801090 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:19.801065    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk"
Nov 04 01:52:19.892611 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:19.892575    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-kube-api-access-dtgpc\" (UniqueName: \"kubernetes.io/projected/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-kube-api-access-dtgpc\") pod \"d949886a-cd12-48ed-9d9a-6fb9e6df4fb8\" (UID: \"d949886a-cd12-48ed-9d9a-6fb9e6df4fb8\") "
Nov 04 01:52:19.892794 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:19.892766    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-maas-api-openshift-ca\") pod \"d949886a-cd12-48ed-9d9a-6fb9e6df4fb8\" (UID: \"d949886a-cd12-48ed-9d9a-6fb9e6df4fb8\") "
Nov 04 01:52:19.892858 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:19.892838    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-maas-api-xks-ca\") pod \"d949886a-cd12-48ed-9d9a-6fb9e6df4fb8\" (UID: \"d949886a-cd12-48ed-9d9a-6fb9e6df4fb8\") "
Nov 04 01:52:19.893111 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:19.893082    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-maas-api-openshift-ca" pod "d949886a-cd12-48ed-9d9a-6fb9e6df4fb8" (UID: "d949886a-cd12-48ed-9d9a-6fb9e6df4fb8"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:52:19.893282 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:19.893268    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-maas-api-openshift-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:19.894739 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:19.894717    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-kube-api-access-dtgpc" pod "d949886a-cd12-48ed-9d9a-6fb9e6df4fb8" (UID: "d949886a-cd12-48ed-9d9a-6fb9e6df4fb8"). InnerVolumeSpecName "kube-api-access-dtgpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:52:19.894857 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:19.894837    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-maas-api-xks-ca" pod "d949886a-cd12-48ed-9d9a-6fb9e6df4fb8" (UID: "d949886a-cd12-48ed-9d9a-6fb9e6df4fb8"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:52:19.994766 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:19.994729    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-maas-api-xks-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:19.994766 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:19.994759    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtgpc\" (UniqueName: \"kubernetes.io/projected/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8-kube-api-access-dtgpc\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:20.653640 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:20.653605    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk" event={"ID":"d949886a-cd12-48ed-9d9a-6fb9e6df4fb8","Type":"ContainerDied","Data":"d95b59ecd014aa304b7cdd9d56f0ff09fa7094d8226788e78a85e797c58bb62e"}
Nov 04 01:52:20.653847 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:20.653644    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk"
Nov 04 01:52:20.653847 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:20.653651    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d95b59ecd014aa304b7cdd9d56f0ff09fa7094d8226788e78a85e797c58bb62e"
Nov 04 01:52:23.335677 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.335644    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"]
Nov 04 01:52:23.336182 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.336165    6116 cpu_manager.go:414] "containerMap: removing container" podUID="d949886a-cd12-48ed-9d9a-6fb9e6df4fb8" containerName="revoke-keys"
Nov 04 01:52:23.336182 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.336181    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d949886a-cd12-48ed-9d9a-6fb9e6df4fb8" containerName="revoke-keys"
Nov 04 01:52:23.336302 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.336293    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="d949886a-cd12-48ed-9d9a-6fb9e6df4fb8" containerName="revoke-keys"
Nov 04 01:52:23.339966 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.339945    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.392181 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.392144    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-plld6\""
Nov 04 01:52:23.441253 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.441219    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.441440 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.441316    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fde313ca-17ff-426b-814f-f27b28fd8023-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.441440 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.441357    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvxk8\" (UniqueName: \"kubernetes.io/projected/fde313ca-17ff-426b-814f-f27b28fd8023-kube-api-access-cvxk8\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.441562 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.441521    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.441693 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.441659    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fde313ca-17ff-426b-814f-f27b28fd8023-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.441893 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.441757    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fde313ca-17ff-426b-814f-f27b28fd8023-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.441893 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.441806    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.441893 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.441851    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.441893 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.441872    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.445593 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.445569    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"]
Nov 04 01:52:23.543370 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.543337    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fde313ca-17ff-426b-814f-f27b28fd8023-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.543370 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.543374    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.543645 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.543403    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.543645 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.543427    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.543645 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.543559    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.543645 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.543631    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fde313ca-17ff-426b-814f-f27b28fd8023-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.543848 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.543746    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvxk8\" (UniqueName: \"kubernetes.io/projected/fde313ca-17ff-426b-814f-f27b28fd8023-kube-api-access-cvxk8\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.543848 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.543804    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.543954 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.543885    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.543954 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.543907    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fde313ca-17ff-426b-814f-f27b28fd8023-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.543954 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.543815    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.544266 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.544245    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fde313ca-17ff-426b-814f-f27b28fd8023-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.544330 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.544312    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.544594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.544564    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.545874 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.545854    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.546103 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.546084    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fde313ca-17ff-426b-814f-f27b28fd8023-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.605241 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.605170    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fde313ca-17ff-426b-814f-f27b28fd8023-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.605368 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.605256    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvxk8\" (UniqueName: \"kubernetes.io/projected/fde313ca-17ff-426b-814f-f27b28fd8023-kube-api-access-cvxk8\") pod \"maas-default-gateway-openshift-default-778b7b5c47-nbx74\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:23.654213 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:23.654178    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:24.086694 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:24.086667    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"]
Nov 04 01:52:24.087840 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:52:24.087810    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfde313ca_17ff_426b_814f_f27b28fd8023.slice/crio-96d6f23edbdb36cd87a04dd1ceec34572c5a3f019e57a8e32905e2214dd05061 WatchSource:0}: Error finding container 96d6f23edbdb36cd87a04dd1ceec34572c5a3f019e57a8e32905e2214dd05061: Status 404 returned error can't find the container with id 96d6f23edbdb36cd87a04dd1ceec34572c5a3f019e57a8e32905e2214dd05061
Nov 04 01:52:24.090253 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:24.090220    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:24.090342 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:24.090287    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:24.090342 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:24.090314    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:24.703195 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:24.703162    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74" event={"ID":"fde313ca-17ff-426b-814f-f27b28fd8023","Type":"ContainerStarted","Data":"be908c8cbfe3f2be2f0be9f6ea745a15c6645afd4d820d15e40b302eac2c5dcf"}
Nov 04 01:52:24.703727 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:24.703209    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74" event={"ID":"fde313ca-17ff-426b-814f-f27b28fd8023","Type":"ContainerStarted","Data":"96d6f23edbdb36cd87a04dd1ceec34572c5a3f019e57a8e32905e2214dd05061"}
Nov 04 01:52:24.829424 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:24.829367    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74" podStartSLOduration=1.829347643 podStartE2EDuration="1.829347643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:52:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:52:24.826269392 +0000 UTC m=+1029.947546917" watchObservedRunningTime="2026-11-04 01:52:24.829347643 +0000 UTC m=+1029.950625171"
Nov 04 01:52:25.661903 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.661865    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:25.667626 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.667575    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-nbx74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 01:52:25.667898 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.667864    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74" podUID="fde313ca-17ff-426b-814f-f27b28fd8023" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 01:52:25.786625 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.786596    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"]
Nov 04 01:52:25.790687 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.790669    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.884913 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.884879    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.885115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.884946    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.885115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.885070    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg5ff\" (UniqueName: \"kubernetes.io/projected/93219663-8259-41a2-9b98-8488f493683d-kube-api-access-zg5ff\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.885241 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.885139    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93219663-8259-41a2-9b98-8488f493683d-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.885241 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.885179    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.885241 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.885208    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93219663-8259-41a2-9b98-8488f493683d-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.885394 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.885282    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.885394 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.885301    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.885394 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.885370    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93219663-8259-41a2-9b98-8488f493683d-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.986885 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.986787    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg5ff\" (UniqueName: \"kubernetes.io/projected/93219663-8259-41a2-9b98-8488f493683d-kube-api-access-zg5ff\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.987067 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.986889    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93219663-8259-41a2-9b98-8488f493683d-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.987067 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.986930    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.987067 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.986952    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93219663-8259-41a2-9b98-8488f493683d-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.987067 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.987007    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.987067 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.987030    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.987325 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.987069    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93219663-8259-41a2-9b98-8488f493683d-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.987430 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.987400    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.987567 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.987428    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.987567 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.987537    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.987567 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.987543    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.987735 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.987713    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.987950 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.987787    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.988131 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.987994    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93219663-8259-41a2-9b98-8488f493683d-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.989727 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.989694    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.990028 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.990006    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93219663-8259-41a2-9b98-8488f493683d-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:25.995665 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:25.995640    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"]
Nov 04 01:52:26.072684 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:26.072652    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg5ff\" (UniqueName: \"kubernetes.io/projected/93219663-8259-41a2-9b98-8488f493683d-kube-api-access-zg5ff\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:26.072920 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:26.072652    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93219663-8259-41a2-9b98-8488f493683d-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:26.105415 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:26.105382    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:26.569485 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:26.569430    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"]
Nov 04 01:52:26.596352 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:52:26.596313    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93219663_8259_41a2_9b98_8488f493683d.slice/crio-c2905fc40480fafa18f6364fc556e173b93db5f266e3863a1aebd3f3c1c99170 WatchSource:0}: Error finding container c2905fc40480fafa18f6364fc556e173b93db5f266e3863a1aebd3f3c1c99170: Status 404 returned error can't find the container with id c2905fc40480fafa18f6364fc556e173b93db5f266e3863a1aebd3f3c1c99170
Nov 04 01:52:26.598886 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:26.598849    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:26.598989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:26.598931    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:26.598989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:26.598961    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:26.660483 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:26.656975    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-nbx74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 01:52:26.660483 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:26.657036    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74" podUID="fde313ca-17ff-426b-814f-f27b28fd8023" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 01:52:26.718798 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:26.718523    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9" event={"ID":"93219663-8259-41a2-9b98-8488f493683d","Type":"ContainerStarted","Data":"7c68a1fa56695d720311617cf3072c1131f987c1cf0e1bf4ed24787aa602555c"}
Nov 04 01:52:26.718798 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:26.718583    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9" event={"ID":"93219663-8259-41a2-9b98-8488f493683d","Type":"ContainerStarted","Data":"c2905fc40480fafa18f6364fc556e173b93db5f266e3863a1aebd3f3c1c99170"}
Nov 04 01:52:26.844025 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:26.843908    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9" podStartSLOduration=1.843890231 podStartE2EDuration="1.843890231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:52:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:52:26.840294859 +0000 UTC m=+1031.961572387" watchObservedRunningTime="2026-11-04 01:52:26.843890231 +0000 UTC m=+1031.965167759"
Nov 04 01:52:27.106224 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:27.106139    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:27.107845 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:27.107823    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 01:52:27.107966 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:27.107873    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9" podUID="93219663-8259-41a2-9b98-8488f493683d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 01:52:27.668926 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:27.668884    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-nbx74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 01:52:27.669226 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:27.669196    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74" podUID="fde313ca-17ff-426b-814f-f27b28fd8023" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 01:52:28.106121 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:28.106019    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 01:52:28.106121 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:28.106078    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9" podUID="93219663-8259-41a2-9b98-8488f493683d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 01:52:28.670486 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:28.662894    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-nbx74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 01:52:28.670486 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:28.662961    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74" podUID="fde313ca-17ff-426b-814f-f27b28fd8023" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 01:52:29.106008 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:29.105920    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 01:52:29.106008 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:29.105972    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9" podUID="93219663-8259-41a2-9b98-8488f493683d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 01:52:29.664939 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:29.664901    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-nbx74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 01:52:29.665233 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:29.665202    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74" podUID="fde313ca-17ff-426b-814f-f27b28fd8023" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 01:52:30.106046 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:30.105963    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 01:52:30.106046 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:30.106014    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9" podUID="93219663-8259-41a2-9b98-8488f493683d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 01:52:30.666481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:30.663831    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-nbx74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 01:52:30.666481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:30.663902    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74" podUID="fde313ca-17ff-426b-814f-f27b28fd8023" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 01:52:31.106114 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:31.106029    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 01:52:31.106114 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:31.106087    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9" podUID="93219663-8259-41a2-9b98-8488f493683d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 01:52:31.654923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:31.654881    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-nbx74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 01:52:31.655222 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:31.655191    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74" podUID="fde313ca-17ff-426b-814f-f27b28fd8023" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 01:52:32.106100 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:32.106007    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 01:52:32.106100 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:32.106079    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9" podUID="93219663-8259-41a2-9b98-8488f493683d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 01:52:32.832340 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:32.832305    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m"]
Nov 04 01:52:32.839997 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:32.839961    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m"
Nov 04 01:52:32.874212 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:32.874167    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lssx2\" (UniqueName: \"kubernetes.io/projected/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-kube-api-access-lssx2\") pod \"maas-api-revoke-sub-497ccb8c-nr92m\" (UID: \"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m"
Nov 04 01:52:32.874380 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:32.874331    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-497ccb8c-nr92m\" (UID: \"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m"
Nov 04 01:52:32.874569 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:32.874550    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-maas-api-xks-ca\") pod \"maas-api-revoke-sub-497ccb8c-nr92m\" (UID: \"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m"
Nov 04 01:52:32.893990 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:32.893950    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m"]
Nov 04 01:52:32.939605 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:32.939568    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 01:52:32.939857 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:32.939607    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:52:32.939991 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:32.939696    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:52:32.941933 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:32.941906    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bj8b6\""
Nov 04 01:52:32.976072 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:32.976040    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lssx2\" (UniqueName: \"kubernetes.io/projected/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-kube-api-access-lssx2\") pod \"maas-api-revoke-sub-497ccb8c-nr92m\" (UID: \"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m"
Nov 04 01:52:32.976256 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:32.976137    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-497ccb8c-nr92m\" (UID: \"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m"
Nov 04 01:52:32.976393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:32.976355    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-maas-api-xks-ca\") pod \"maas-api-revoke-sub-497ccb8c-nr92m\" (UID: \"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m"
Nov 04 01:52:32.977173 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:32.977133    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-497ccb8c-nr92m\" (UID: \"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m"
Nov 04 01:52:32.979648 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:32.979623    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-maas-api-xks-ca\") pod \"maas-api-revoke-sub-497ccb8c-nr92m\" (UID: \"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m"
Nov 04 01:52:33.038780 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:33.038740    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lssx2\" (UniqueName: \"kubernetes.io/projected/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-kube-api-access-lssx2\") pod \"maas-api-revoke-sub-497ccb8c-nr92m\" (UID: \"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m"
Nov 04 01:52:33.106883 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:33.106794    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 04 01:52:33.106883 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:33.106861    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9" podUID="93219663-8259-41a2-9b98-8488f493683d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 04 01:52:33.158857 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:33.158796    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m"
Nov 04 01:52:33.320346 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:33.320309    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:33.320961 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:33.320934    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:33.322571 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:33.322548    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:52:33.562758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:33.562718    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m"]
Nov 04 01:52:33.562881 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:52:33.562782    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode04dcc2f_f680_4bd3_9cf4_5b7a00f3cc56.slice/crio-aa389550a1394cc5b693acdad693c0a35714f33f7f84721fda7495a13d856ba2 WatchSource:0}: Error finding container aa389550a1394cc5b693acdad693c0a35714f33f7f84721fda7495a13d856ba2: Status 404 returned error can't find the container with id aa389550a1394cc5b693acdad693c0a35714f33f7f84721fda7495a13d856ba2
Nov 04 01:52:33.881526 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:33.881488    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m" event={"ID":"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56","Type":"ContainerStarted","Data":"a107736f1f94cda6ddb67679c9575a0556043a82a013a7e872451668a22c6cef"}
Nov 04 01:52:33.881924 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:33.881560    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m" event={"ID":"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56","Type":"ContainerStarted","Data":"aa389550a1394cc5b693acdad693c0a35714f33f7f84721fda7495a13d856ba2"}
Nov 04 01:52:33.997730 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:33.997685    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m" podStartSLOduration=1.997671262 podStartE2EDuration="1.997671262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:52:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:52:33.995960803 +0000 UTC m=+1039.117238343" watchObservedRunningTime="2026-11-04 01:52:33.997671262 +0000 UTC m=+1039.118948766"
Nov 04 01:52:34.144022 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:34.143934    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:34.144267 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:34.144240    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:34.145623 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:34.145599    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:52:34.888650 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:34.888564    6116 generic.go:356] "Generic (PLEG): container finished" podID="e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56" containerID="a107736f1f94cda6ddb67679c9575a0556043a82a013a7e872451668a22c6cef" exitCode=0
Nov 04 01:52:34.889074 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:34.888651    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m" event={"ID":"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56","Type":"ContainerDied","Data":"a107736f1f94cda6ddb67679c9575a0556043a82a013a7e872451668a22c6cef"}
Nov 04 01:52:36.040018 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:36.039989    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m"
Nov 04 01:52:36.120587 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:36.120550    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-maas-api-openshift-ca\") pod \"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56\" (UID: \"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56\") "
Nov 04 01:52:36.120781 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:36.120769    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-kube-api-access-lssx2\" (UniqueName: \"kubernetes.io/projected/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-kube-api-access-lssx2\") pod \"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56\" (UID: \"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56\") "
Nov 04 01:52:36.120904 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:36.120889    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-maas-api-xks-ca\") pod \"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56\" (UID: \"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56\") "
Nov 04 01:52:36.120967 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:36.120904    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-maas-api-openshift-ca" pod "e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56" (UID: "e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:52:36.121856 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:36.121825    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-maas-api-openshift-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:36.123099 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:36.123067    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-maas-api-xks-ca" pod "e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56" (UID: "e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:52:36.123177 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:36.123113    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-kube-api-access-lssx2" pod "e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56" (UID: "e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56"). InnerVolumeSpecName "kube-api-access-lssx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:52:36.222568 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:36.222529    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lssx2\" (UniqueName: \"kubernetes.io/projected/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-kube-api-access-lssx2\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:36.222568 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:36.222561    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56-maas-api-xks-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:36.901688 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:36.901652    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m" event={"ID":"e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56","Type":"ContainerDied","Data":"aa389550a1394cc5b693acdad693c0a35714f33f7f84721fda7495a13d856ba2"}
Nov 04 01:52:36.901839 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:36.901692    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m"
Nov 04 01:52:36.901839 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:36.901700    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa389550a1394cc5b693acdad693c0a35714f33f7f84721fda7495a13d856ba2"
Nov 04 01:52:37.195849 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:37.195753    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"]
Nov 04 01:52:38.479415 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.479383    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"]
Nov 04 01:52:38.480038 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.480022    6116 cpu_manager.go:414] "containerMap: removing container" podUID="e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56" containerName="revoke-keys"
Nov 04 01:52:38.480082 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.480039    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56" containerName="revoke-keys"
Nov 04 01:52:38.480139 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.480130    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56" containerName="revoke-keys"
Nov 04 01:52:38.483637 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.483614    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.529925 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.529888    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"]
Nov 04 01:52:38.533796 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.533775    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.550317 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.550291    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.550437 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.550415    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e4522dd-634e-4890-8ba5-67676e1575a9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.550521 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.550482    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.550521 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.550510    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.550628 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.550609    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.550662 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.550643    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.550716 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.550667    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.550805 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.550788    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.550864 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.550831    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wwh6\" (UniqueName: \"kubernetes.io/projected/2e4522dd-634e-4890-8ba5-67676e1575a9-kube-api-access-9wwh6\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.585832 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.585798    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"]
Nov 04 01:52:38.640405 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.640366    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"]
Nov 04 01:52:38.651795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.651761    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.651795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.651798    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.652028 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.651831    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e4522dd-634e-4890-8ba5-67676e1575a9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.652093 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652077    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.652151 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652111    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.652151 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652127    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.652774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652155    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.652774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652228    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.652774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652263    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.652774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652297    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.652774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652329    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.652774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652374    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.652774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652421    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.652774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652485    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db9kl\" (UniqueName: \"kubernetes.io/projected/dad0a1c4-88b2-40ee-9a4a-8997fc743448-kube-api-access-db9kl\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.652774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652529    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.652774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652565    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.652774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652595    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wwh6\" (UniqueName: \"kubernetes.io/projected/2e4522dd-634e-4890-8ba5-67676e1575a9-kube-api-access-9wwh6\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.652774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652682    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e4522dd-634e-4890-8ba5-67676e1575a9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.652774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652731    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.652774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652768    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.652774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652487    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.653557 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.652841    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.653557 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.653057    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.655012 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.654983    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.655106 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.654982    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.714056 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.713992    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.714197 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.714122    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wwh6\" (UniqueName: \"kubernetes.io/projected/2e4522dd-634e-4890-8ba5-67676e1575a9-kube-api-access-9wwh6\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xml2m\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.753884 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.753798    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.753884 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.753833    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.753884 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.753877    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.754158 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.753899    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.754158 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.753936    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.754158 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.753971    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.754158 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.754020    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.754158 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.754060    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db9kl\" (UniqueName: \"kubernetes.io/projected/dad0a1c4-88b2-40ee-9a4a-8997fc743448-kube-api-access-db9kl\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.754158 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.754110    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.754474 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.754254    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.754474 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.754320    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.754474 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.754334    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.754688 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.754652    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.755039 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.754982    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.756630 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.756609    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.756768 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.756751    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.797357 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.797329    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:38.815170 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.815148    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.815298 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.815279    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-db9kl\" (UniqueName: \"kubernetes.io/projected/dad0a1c4-88b2-40ee-9a4a-8997fc743448-kube-api-access-db9kl\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m9d6q\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:38.846603 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:38.846575    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:39.230588 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:39.230549    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"]
Nov 04 01:52:39.268092 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:52:39.268066    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e4522dd_634e_4890_8ba5_67676e1575a9.slice/crio-3bbe4b9bd7ada669fd51976ac0b6105e1208856915ca897de8e72f74ebed7fd5 WatchSource:0}: Error finding container 3bbe4b9bd7ada669fd51976ac0b6105e1208856915ca897de8e72f74ebed7fd5: Status 404 returned error can't find the container with id 3bbe4b9bd7ada669fd51976ac0b6105e1208856915ca897de8e72f74ebed7fd5
Nov 04 01:52:39.270350 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:39.270316    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:39.270423 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:39.270383    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:39.270423 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:39.270412    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:39.284489 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:39.284464    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"]
Nov 04 01:52:39.286134 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:52:39.286106    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddad0a1c4_88b2_40ee_9a4a_8997fc743448.slice/crio-270afcb333e90d3a0d198a57616d60b6467a607240a11490ff9028902a8f68ca WatchSource:0}: Error finding container 270afcb333e90d3a0d198a57616d60b6467a607240a11490ff9028902a8f68ca: Status 404 returned error can't find the container with id 270afcb333e90d3a0d198a57616d60b6467a607240a11490ff9028902a8f68ca
Nov 04 01:52:39.288639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:39.288601    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:39.288744 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:39.288672    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:39.288744 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:39.288701    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:52:39.929779 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:39.929720    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m" event={"ID":"2e4522dd-634e-4890-8ba5-67676e1575a9","Type":"ContainerStarted","Data":"0f7725dd358d69003ee9dda123aeb545c2d4c2bcb28ebf09214d388797e9f459"}
Nov 04 01:52:39.930285 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:39.929798    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m" event={"ID":"2e4522dd-634e-4890-8ba5-67676e1575a9","Type":"ContainerStarted","Data":"3bbe4b9bd7ada669fd51976ac0b6105e1208856915ca897de8e72f74ebed7fd5"}
Nov 04 01:52:39.932481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:39.932402    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q" event={"ID":"dad0a1c4-88b2-40ee-9a4a-8997fc743448","Type":"ContainerStarted","Data":"ccdbf2ba8bdbb20e844ccebe6c48db078ac7742903ae99ce741835afd07f99d6"}
Nov 04 01:52:39.932481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:39.932471    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q" event={"ID":"dad0a1c4-88b2-40ee-9a4a-8997fc743448","Type":"ContainerStarted","Data":"270afcb333e90d3a0d198a57616d60b6467a607240a11490ff9028902a8f68ca"}
Nov 04 01:52:40.064961 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:40.064893    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m" podStartSLOduration=2.064873056 podStartE2EDuration="2.064873056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:52:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:52:40.063866988 +0000 UTC m=+1045.185144515" watchObservedRunningTime="2026-11-04 01:52:40.064873056 +0000 UTC m=+1045.186150582"
Nov 04 01:52:40.816773 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:40.816733    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:40.818844 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:40.818812    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xml2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 01:52:40.819049 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:40.819024    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m" podUID="2e4522dd-634e-4890-8ba5-67676e1575a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 01:52:40.904305 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:40.904271    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:40.904697 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:40.904675    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-m9d6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 04 01:52:40.904842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:40.904821    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q" podUID="dad0a1c4-88b2-40ee-9a4a-8997fc743448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 04 01:52:41.797859 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:41.797759    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xml2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 01:52:41.797859 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:41.797831    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m" podUID="2e4522dd-634e-4890-8ba5-67676e1575a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 01:52:41.859938 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:41.859904    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-m9d6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 04 01:52:41.860855 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:41.860822    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q" podUID="dad0a1c4-88b2-40ee-9a4a-8997fc743448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 04 01:52:42.023359 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:52:42.023326    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:52:42.813477 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:42.809861    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xml2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 01:52:42.813477 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:42.809934    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m" podUID="2e4522dd-634e-4890-8ba5-67676e1575a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 01:52:42.855469 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:42.849914    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-m9d6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 04 01:52:42.855469 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:42.849968    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q" podUID="dad0a1c4-88b2-40ee-9a4a-8997fc743448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 04 01:52:43.800955 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:43.800918    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xml2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 01:52:43.801265 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:43.801230    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m" podUID="2e4522dd-634e-4890-8ba5-67676e1575a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 01:52:43.878627 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:43.878586    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-m9d6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 04 01:52:43.879199 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:43.879164    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q" podUID="dad0a1c4-88b2-40ee-9a4a-8997fc743448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 04 01:52:44.799475 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:44.798804    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xml2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 01:52:44.799475 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:44.798865    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m" podUID="2e4522dd-634e-4890-8ba5-67676e1575a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 01:52:44.863478 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:44.857791    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-m9d6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 04 01:52:44.863478 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:44.857870    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q" podUID="dad0a1c4-88b2-40ee-9a4a-8997fc743448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 04 01:52:45.798317 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:45.798274    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xml2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 01:52:45.798915 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:45.798339    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m" podUID="2e4522dd-634e-4890-8ba5-67676e1575a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 01:52:45.847989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:45.847819    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-m9d6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 04 01:52:45.847989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:45.847894    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q" podUID="dad0a1c4-88b2-40ee-9a4a-8997fc743448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 04 01:52:46.798807 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:46.798768    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xml2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 01:52:46.799300 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:46.798836    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m" podUID="2e4522dd-634e-4890-8ba5-67676e1575a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 01:52:46.847235 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:46.847180    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-m9d6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 04 01:52:46.847586 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:46.847534    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q" podUID="dad0a1c4-88b2-40ee-9a4a-8997fc743448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 04 01:52:48.449462 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:48.449318    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:48.450052 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:48.449906    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:48.451150 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:48.451130    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:52:48.528602 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:48.528565    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:48.528901 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:48.528882    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:48.529717 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:48.529697    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:52:48.587025 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:48.586971    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q" podStartSLOduration=10.586956231 podStartE2EDuration="10.586956231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:52:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:52:40.19605081 +0000 UTC m=+1045.317328369" watchObservedRunningTime="2026-11-04 01:52:48.586956231 +0000 UTC m=+1053.708233757"
Nov 04 01:52:49.631842 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:49.631811    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774"]
Nov 04 01:52:49.636250 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:49.636230    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774"
Nov 04 01:52:49.690035 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:49.688969    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:52:49.697645 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:49.697601    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774"]
Nov 04 01:52:49.713288 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:49.713249    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a334c719-70fa-4886-bd0f-57d102af0a77-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7d3605df-hx774\" (UID: \"a334c719-70fa-4886-bd0f-57d102af0a77\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774"
Nov 04 01:52:49.713716 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:49.713690    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a334c719-70fa-4886-bd0f-57d102af0a77-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7d3605df-hx774\" (UID: \"a334c719-70fa-4886-bd0f-57d102af0a77\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774"
Nov 04 01:52:49.713945 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:49.713926    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcr85\" (UniqueName: \"kubernetes.io/projected/a334c719-70fa-4886-bd0f-57d102af0a77-kube-api-access-gcr85\") pod \"maas-api-revoke-sub-7d3605df-hx774\" (UID: \"a334c719-70fa-4886-bd0f-57d102af0a77\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774"
Nov 04 01:52:49.738398 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:49.738373    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 01:52:49.738398 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:49.738387    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bj8b6\""
Nov 04 01:52:49.738611 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:49.738387    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:52:49.815752 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:49.815708    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a334c719-70fa-4886-bd0f-57d102af0a77-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7d3605df-hx774\" (UID: \"a334c719-70fa-4886-bd0f-57d102af0a77\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774"
Nov 04 01:52:49.815928 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:49.815843    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a334c719-70fa-4886-bd0f-57d102af0a77-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7d3605df-hx774\" (UID: \"a334c719-70fa-4886-bd0f-57d102af0a77\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774"
Nov 04 01:52:49.815928 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:49.815910    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcr85\" (UniqueName: \"kubernetes.io/projected/a334c719-70fa-4886-bd0f-57d102af0a77-kube-api-access-gcr85\") pod \"maas-api-revoke-sub-7d3605df-hx774\" (UID: \"a334c719-70fa-4886-bd0f-57d102af0a77\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774"
Nov 04 01:52:49.816412 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:49.816387    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a334c719-70fa-4886-bd0f-57d102af0a77-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7d3605df-hx774\" (UID: \"a334c719-70fa-4886-bd0f-57d102af0a77\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774"
Nov 04 01:52:49.818220 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:49.818199    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a334c719-70fa-4886-bd0f-57d102af0a77-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7d3605df-hx774\" (UID: \"a334c719-70fa-4886-bd0f-57d102af0a77\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774"
Nov 04 01:52:49.880237 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:49.880203    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcr85\" (UniqueName: \"kubernetes.io/projected/a334c719-70fa-4886-bd0f-57d102af0a77-kube-api-access-gcr85\") pod \"maas-api-revoke-sub-7d3605df-hx774\" (UID: \"a334c719-70fa-4886-bd0f-57d102af0a77\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774"
Nov 04 01:52:49.953068 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:49.953037    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774"
Nov 04 01:52:50.445206 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:50.445177    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774"]
Nov 04 01:52:51.101473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:51.100865    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774" event={"ID":"a334c719-70fa-4886-bd0f-57d102af0a77","Type":"ContainerStarted","Data":"4849ed0e3ef8d0fa8bcdc445f80a484d29e3a3b845a3489c1b82e82ba2f621cb"}
Nov 04 01:52:51.101473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:51.100936    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774" event={"ID":"a334c719-70fa-4886-bd0f-57d102af0a77","Type":"ContainerStarted","Data":"28f3917d8b972dd22b60150e0f4872ad36bb8b75930eb0e7b919e30dee77432d"}
Nov 04 01:52:51.940943 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:51.940907    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x"]
Nov 04 01:52:52.108268 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.108237    6116 generic.go:356] "Generic (PLEG): container finished" podID="a334c719-70fa-4886-bd0f-57d102af0a77" containerID="4849ed0e3ef8d0fa8bcdc445f80a484d29e3a3b845a3489c1b82e82ba2f621cb" exitCode=0
Nov 04 01:52:52.108687 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.108330    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774" event={"ID":"a334c719-70fa-4886-bd0f-57d102af0a77","Type":"ContainerDied","Data":"4849ed0e3ef8d0fa8bcdc445f80a484d29e3a3b845a3489c1b82e82ba2f621cb"}
Nov 04 01:52:52.216114 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.216025    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk" podUID="a242d84a-50e6-4dbc-8179-60f694e0b80c" containerName="main" containerID="cri-o://043c9570f32c2e61141af943fb2994fb551e0c9384e3be3bb5e3cd47fa568ac1" gracePeriod=45
Nov 04 01:52:52.490000 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.489971    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:52:52.590614 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.590567    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-model-cache\") pod \"a242d84a-50e6-4dbc-8179-60f694e0b80c\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") "
Nov 04 01:52:52.590824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.590638    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a242d84a-50e6-4dbc-8179-60f694e0b80c-kube-api-access-dsnwd\" (UniqueName: \"kubernetes.io/projected/a242d84a-50e6-4dbc-8179-60f694e0b80c-kube-api-access-dsnwd\") pod \"a242d84a-50e6-4dbc-8179-60f694e0b80c\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") "
Nov 04 01:52:52.590824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.590675    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-home\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-home\") pod \"a242d84a-50e6-4dbc-8179-60f694e0b80c\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") "
Nov 04 01:52:52.590824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.590797    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-tmp-dir\") pod \"a242d84a-50e6-4dbc-8179-60f694e0b80c\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") "
Nov 04 01:52:52.590992 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.590862    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-dshm\") pod \"a242d84a-50e6-4dbc-8179-60f694e0b80c\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") "
Nov 04 01:52:52.590992 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.590890    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-model-cache" pod "a242d84a-50e6-4dbc-8179-60f694e0b80c" (UID: "a242d84a-50e6-4dbc-8179-60f694e0b80c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:52:52.590992 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.590896    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-home" pod "a242d84a-50e6-4dbc-8179-60f694e0b80c" (UID: "a242d84a-50e6-4dbc-8179-60f694e0b80c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:52:52.590992 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.590941    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a242d84a-50e6-4dbc-8179-60f694e0b80c-tls-certs\" (UniqueName: \"kubernetes.io/secret/a242d84a-50e6-4dbc-8179-60f694e0b80c-tls-certs\") pod \"a242d84a-50e6-4dbc-8179-60f694e0b80c\" (UID: \"a242d84a-50e6-4dbc-8179-60f694e0b80c\") "
Nov 04 01:52:52.591135 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.591012    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-tmp-dir" pod "a242d84a-50e6-4dbc-8179-60f694e0b80c" (UID: "a242d84a-50e6-4dbc-8179-60f694e0b80c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:52:52.591400 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.591382    6116 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-tmp-dir\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:52.591690 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.591406    6116 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-model-cache\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:52.591690 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.591419    6116 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-home\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:52.592987 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.592957    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-dshm" pod "a242d84a-50e6-4dbc-8179-60f694e0b80c" (UID: "a242d84a-50e6-4dbc-8179-60f694e0b80c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:52:52.593083 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.592991    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a242d84a-50e6-4dbc-8179-60f694e0b80c-tls-certs" pod "a242d84a-50e6-4dbc-8179-60f694e0b80c" (UID: "a242d84a-50e6-4dbc-8179-60f694e0b80c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:52:52.593083 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.593000    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a242d84a-50e6-4dbc-8179-60f694e0b80c-kube-api-access-dsnwd" pod "a242d84a-50e6-4dbc-8179-60f694e0b80c" (UID: "a242d84a-50e6-4dbc-8179-60f694e0b80c"). InnerVolumeSpecName "kube-api-access-dsnwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:52:52.693080 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.693045    6116 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a242d84a-50e6-4dbc-8179-60f694e0b80c-dshm\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:52.693080 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.693079    6116 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a242d84a-50e6-4dbc-8179-60f694e0b80c-tls-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:52.693080 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:52.693095    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsnwd\" (UniqueName: \"kubernetes.io/projected/a242d84a-50e6-4dbc-8179-60f694e0b80c-kube-api-access-dsnwd\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:53.116650 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.116618    6116 generic.go:356] "Generic (PLEG): container finished" podID="a242d84a-50e6-4dbc-8179-60f694e0b80c" containerID="043c9570f32c2e61141af943fb2994fb551e0c9384e3be3bb5e3cd47fa568ac1" exitCode=0
Nov 04 01:52:53.117099 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.116665    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk" event={"ID":"a242d84a-50e6-4dbc-8179-60f694e0b80c","Type":"ContainerDied","Data":"043c9570f32c2e61141af943fb2994fb551e0c9384e3be3bb5e3cd47fa568ac1"}
Nov 04 01:52:53.117099 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.116680    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"
Nov 04 01:52:53.117099 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.116717    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk" event={"ID":"a242d84a-50e6-4dbc-8179-60f694e0b80c","Type":"ContainerDied","Data":"cf7aa3dd69a944cd0602c610d4b4202b73691f17b8dbe7a61d658054532b965f"}
Nov 04 01:52:53.117099 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.116739    6116 scope.go:122] "RemoveContainer" containerID="043c9570f32c2e61141af943fb2994fb551e0c9384e3be3bb5e3cd47fa568ac1"
Nov 04 01:52:53.129210 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.129186    6116 scope.go:122] "RemoveContainer" containerID="043c9570f32c2e61141af943fb2994fb551e0c9384e3be3bb5e3cd47fa568ac1"
Nov 04 01:52:53.129525 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:52:53.129506    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"043c9570f32c2e61141af943fb2994fb551e0c9384e3be3bb5e3cd47fa568ac1\": container with ID starting with 043c9570f32c2e61141af943fb2994fb551e0c9384e3be3bb5e3cd47fa568ac1 not found: ID does not exist" containerID="043c9570f32c2e61141af943fb2994fb551e0c9384e3be3bb5e3cd47fa568ac1"
Nov 04 01:52:53.129589 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.129537    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"043c9570f32c2e61141af943fb2994fb551e0c9384e3be3bb5e3cd47fa568ac1"} err="failed to get container status \"043c9570f32c2e61141af943fb2994fb551e0c9384e3be3bb5e3cd47fa568ac1\": rpc error: code = NotFound desc = could not find container \"043c9570f32c2e61141af943fb2994fb551e0c9384e3be3bb5e3cd47fa568ac1\": container with ID starting with 043c9570f32c2e61141af943fb2994fb551e0c9384e3be3bb5e3cd47fa568ac1 not found: ID does not exist"
Nov 04 01:52:53.290536 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.290410    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"]
Nov 04 01:52:53.339476 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.339139    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-f4d202/simulator-w4-f4d202-kserve-849dc49777-7wdsk"]
Nov 04 01:52:53.378197 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.378163    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774"
Nov 04 01:52:53.505535 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.505504    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a334c719-70fa-4886-bd0f-57d102af0a77-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a334c719-70fa-4886-bd0f-57d102af0a77-maas-api-xks-ca\") pod \"a334c719-70fa-4886-bd0f-57d102af0a77\" (UID: \"a334c719-70fa-4886-bd0f-57d102af0a77\") "
Nov 04 01:52:53.505729 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.505560    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a334c719-70fa-4886-bd0f-57d102af0a77-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a334c719-70fa-4886-bd0f-57d102af0a77-maas-api-openshift-ca\") pod \"a334c719-70fa-4886-bd0f-57d102af0a77\" (UID: \"a334c719-70fa-4886-bd0f-57d102af0a77\") "
Nov 04 01:52:53.505729 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.505657    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a334c719-70fa-4886-bd0f-57d102af0a77-kube-api-access-gcr85\" (UniqueName: \"kubernetes.io/projected/a334c719-70fa-4886-bd0f-57d102af0a77-kube-api-access-gcr85\") pod \"a334c719-70fa-4886-bd0f-57d102af0a77\" (UID: \"a334c719-70fa-4886-bd0f-57d102af0a77\") "
Nov 04 01:52:53.505953 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.505924    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a334c719-70fa-4886-bd0f-57d102af0a77-maas-api-openshift-ca" pod "a334c719-70fa-4886-bd0f-57d102af0a77" (UID: "a334c719-70fa-4886-bd0f-57d102af0a77"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:52:53.506214 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.506186    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a334c719-70fa-4886-bd0f-57d102af0a77-maas-api-openshift-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:53.510643 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.510613    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a334c719-70fa-4886-bd0f-57d102af0a77-kube-api-access-gcr85" pod "a334c719-70fa-4886-bd0f-57d102af0a77" (UID: "a334c719-70fa-4886-bd0f-57d102af0a77"). InnerVolumeSpecName "kube-api-access-gcr85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:52:53.510972 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.510950    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a334c719-70fa-4886-bd0f-57d102af0a77-maas-api-xks-ca" pod "a334c719-70fa-4886-bd0f-57d102af0a77" (UID: "a334c719-70fa-4886-bd0f-57d102af0a77"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:52:53.607790 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.607699    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcr85\" (UniqueName: \"kubernetes.io/projected/a334c719-70fa-4886-bd0f-57d102af0a77-kube-api-access-gcr85\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:53.607790 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:53.607730    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a334c719-70fa-4886-bd0f-57d102af0a77-maas-api-xks-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:52:54.129096 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:54.129069    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774"
Nov 04 01:52:54.129586 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:54.129119    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774" event={"ID":"a334c719-70fa-4886-bd0f-57d102af0a77","Type":"ContainerDied","Data":"28f3917d8b972dd22b60150e0f4872ad36bb8b75930eb0e7b919e30dee77432d"}
Nov 04 01:52:54.129586 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:54.129202    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28f3917d8b972dd22b60150e0f4872ad36bb8b75930eb0e7b919e30dee77432d"
Nov 04 01:52:55.028354 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:55.028309    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a242d84a-50e6-4dbc-8179-60f694e0b80c" path="/var/lib/kubelet/pods/a242d84a-50e6-4dbc-8179-60f694e0b80c/volumes"
Nov 04 01:52:56.941623 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:56.941563    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x" podUID="af11b6f0-9314-4cb7-ac0d-76ecf4bb665d" containerName="payload-pre-processing" containerID="cri-o://5d1fd3747cb3e01a6bd088e5fd85d601802f79ab944f019bf0ce375afcd152a8" gracePeriod=25
Nov 04 01:52:57.954264 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:52:57.954230    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x" podUID="af11b6f0-9314-4cb7-ac0d-76ecf4bb665d" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:53:06.161184 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:06.161142    6116 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-84cffd7d1248: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:06.161184 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:06.161181    6116 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:06.161722 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:06.161270    6116 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials podName:233cb7a1-2c92-4544-afe5-5aae953288e1 nodeName:}" failed. No retries permitted until 2026-11-04 01:53:06.661248181 +0000 UTC m=+1071.782525709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials") pod "payload-processing-external-model-5497dccdcc-bbs4k" (UID: "233cb7a1-2c92-4544-afe5-5aae953288e1") : secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:06.668301 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:06.668264    6116 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-84cffd7d1248: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:06.668301 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:06.668295    6116 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:06.668587 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:06.668373    6116 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials podName:233cb7a1-2c92-4544-afe5-5aae953288e1 nodeName:}" failed. No retries permitted until 2026-11-04 01:53:07.668357183 +0000 UTC m=+1072.789634691 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials") pod "payload-processing-external-model-5497dccdcc-bbs4k" (UID: "233cb7a1-2c92-4544-afe5-5aae953288e1") : secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:07.681611 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:07.681568    6116 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-84cffd7d1248: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:07.681611 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:07.681600    6116 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:07.682154 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:07.681665    6116 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials podName:233cb7a1-2c92-4544-afe5-5aae953288e1 nodeName:}" failed. No retries permitted until 2026-11-04 01:53:09.681647951 +0000 UTC m=+1074.802925472 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials") pod "payload-processing-external-model-5497dccdcc-bbs4k" (UID: "233cb7a1-2c92-4544-afe5-5aae953288e1") : secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:07.954620 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:07.954528    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x" podUID="af11b6f0-9314-4cb7-ac0d-76ecf4bb665d" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:53:09.706172 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:09.706137    6116 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-84cffd7d1248: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:09.706172 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:09.706163    6116 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:09.706667 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:09.706219    6116 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials podName:233cb7a1-2c92-4544-afe5-5aae953288e1 nodeName:}" failed. No retries permitted until 2026-11-04 01:53:13.706204509 +0000 UTC m=+1078.827482014 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials") pod "payload-processing-external-model-5497dccdcc-bbs4k" (UID: "233cb7a1-2c92-4544-afe5-5aae953288e1") : secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:11.993034 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:11.992998    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:53:13.756619 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:13.756584    6116 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-84cffd7d1248: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:13.756619 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:13.756610    6116 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:13.757036 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:13.756674    6116 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials podName:233cb7a1-2c92-4544-afe5-5aae953288e1 nodeName:}" failed. No retries permitted until 2026-11-04 01:53:21.756660148 +0000 UTC m=+1086.877937652 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials") pod "payload-processing-external-model-5497dccdcc-bbs4k" (UID: "233cb7a1-2c92-4544-afe5-5aae953288e1") : secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:17.204215 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:17.204138    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x"
Nov 04 01:53:17.277385 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:17.277351    6116 generic.go:356] "Generic (PLEG): container finished" podID="af11b6f0-9314-4cb7-ac0d-76ecf4bb665d" containerID="5d1fd3747cb3e01a6bd088e5fd85d601802f79ab944f019bf0ce375afcd152a8" exitCode=0
Nov 04 01:53:17.277580 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:17.277398    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x"
Nov 04 01:53:17.277580 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:17.277435    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x" event={"ID":"af11b6f0-9314-4cb7-ac0d-76ecf4bb665d","Type":"ContainerDied","Data":"5d1fd3747cb3e01a6bd088e5fd85d601802f79ab944f019bf0ce375afcd152a8"}
Nov 04 01:53:17.277580 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:17.277503    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x" event={"ID":"af11b6f0-9314-4cb7-ac0d-76ecf4bb665d","Type":"ContainerDied","Data":"bcc89b9af7b6acd688bbe7c945abd7e2a9bbc96f3c98da5fad58dc4571478387"}
Nov 04 01:53:17.277580 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:17.277527    6116 scope.go:122] "RemoveContainer" containerID="5d1fd3747cb3e01a6bd088e5fd85d601802f79ab944f019bf0ce375afcd152a8"
Nov 04 01:53:17.287052 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:17.287029    6116 scope.go:122] "RemoveContainer" containerID="5d1fd3747cb3e01a6bd088e5fd85d601802f79ab944f019bf0ce375afcd152a8"
Nov 04 01:53:17.287370 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:17.287343    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d1fd3747cb3e01a6bd088e5fd85d601802f79ab944f019bf0ce375afcd152a8\": container with ID starting with 5d1fd3747cb3e01a6bd088e5fd85d601802f79ab944f019bf0ce375afcd152a8 not found: ID does not exist" containerID="5d1fd3747cb3e01a6bd088e5fd85d601802f79ab944f019bf0ce375afcd152a8"
Nov 04 01:53:17.287605 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:17.287379    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d1fd3747cb3e01a6bd088e5fd85d601802f79ab944f019bf0ce375afcd152a8"} err="failed to get container status \"5d1fd3747cb3e01a6bd088e5fd85d601802f79ab944f019bf0ce375afcd152a8\": rpc error: code = NotFound desc = could not find container \"5d1fd3747cb3e01a6bd088e5fd85d601802f79ab944f019bf0ce375afcd152a8\": container with ID starting with 5d1fd3747cb3e01a6bd088e5fd85d601802f79ab944f019bf0ce375afcd152a8 not found: ID does not exist"
Nov 04 01:53:17.303358 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:17.303336    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af11b6f0-9314-4cb7-ac0d-76ecf4bb665d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af11b6f0-9314-4cb7-ac0d-76ecf4bb665d-plugins-config-volume\") pod \"af11b6f0-9314-4cb7-ac0d-76ecf4bb665d\" (UID: \"af11b6f0-9314-4cb7-ac0d-76ecf4bb665d\") "
Nov 04 01:53:17.303572 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:17.303556    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af11b6f0-9314-4cb7-ac0d-76ecf4bb665d-kube-api-access-4dn2f\" (UniqueName: \"kubernetes.io/projected/af11b6f0-9314-4cb7-ac0d-76ecf4bb665d-kube-api-access-4dn2f\") pod \"af11b6f0-9314-4cb7-ac0d-76ecf4bb665d\" (UID: \"af11b6f0-9314-4cb7-ac0d-76ecf4bb665d\") "
Nov 04 01:53:17.303749 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:17.303729    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af11b6f0-9314-4cb7-ac0d-76ecf4bb665d-plugins-config-volume" pod "af11b6f0-9314-4cb7-ac0d-76ecf4bb665d" (UID: "af11b6f0-9314-4cb7-ac0d-76ecf4bb665d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:53:17.304140 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:17.304120    6116 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af11b6f0-9314-4cb7-ac0d-76ecf4bb665d-plugins-config-volume\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:17.305672 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:17.305654    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af11b6f0-9314-4cb7-ac0d-76ecf4bb665d-kube-api-access-4dn2f" pod "af11b6f0-9314-4cb7-ac0d-76ecf4bb665d" (UID: "af11b6f0-9314-4cb7-ac0d-76ecf4bb665d"). InnerVolumeSpecName "kube-api-access-4dn2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:53:17.404914 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:17.404882    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dn2f\" (UniqueName: \"kubernetes.io/projected/af11b6f0-9314-4cb7-ac0d-76ecf4bb665d-kube-api-access-4dn2f\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:17.700165 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:17.700120    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x"]
Nov 04 01:53:17.748196 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:17.748163    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x"]
Nov 04 01:53:19.028384 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:19.028352    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af11b6f0-9314-4cb7-ac0d-76ecf4bb665d" path="/var/lib/kubelet/pods/af11b6f0-9314-4cb7-ac0d-76ecf4bb665d/volumes"
Nov 04 01:53:21.765283 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:21.765244    6116 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-84cffd7d1248: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:21.765283 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:21.765270    6116 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:21.765834 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:21.765325    6116 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials podName:233cb7a1-2c92-4544-afe5-5aae953288e1 nodeName:}" failed. No retries permitted until 2026-11-04 01:53:37.76531067 +0000 UTC m=+1102.886588184 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials") pod "payload-processing-external-model-5497dccdcc-bbs4k" (UID: "233cb7a1-2c92-4544-afe5-5aae953288e1") : secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:22.689795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:22.689759    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:53:23.854040 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.853842    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"]
Nov 04 01:53:23.855132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.855108    6116 cpu_manager.go:414] "containerMap: removing container" podUID="a334c719-70fa-4886-bd0f-57d102af0a77" containerName="revoke-keys"
Nov 04 01:53:23.855132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.855130    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a334c719-70fa-4886-bd0f-57d102af0a77" containerName="revoke-keys"
Nov 04 01:53:23.855366 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.855164    6116 cpu_manager.go:414] "containerMap: removing container" podUID="af11b6f0-9314-4cb7-ac0d-76ecf4bb665d" containerName="payload-pre-processing"
Nov 04 01:53:23.855366 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.855171    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af11b6f0-9314-4cb7-ac0d-76ecf4bb665d" containerName="payload-pre-processing"
Nov 04 01:53:23.855366 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.855214    6116 cpu_manager.go:414] "containerMap: removing container" podUID="a242d84a-50e6-4dbc-8179-60f694e0b80c" containerName="main"
Nov 04 01:53:23.855366 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.855222    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a242d84a-50e6-4dbc-8179-60f694e0b80c" containerName="main"
Nov 04 01:53:23.855603 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.855381    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="a242d84a-50e6-4dbc-8179-60f694e0b80c" containerName="main"
Nov 04 01:53:23.855603 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.855399    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="a334c719-70fa-4886-bd0f-57d102af0a77" containerName="revoke-keys"
Nov 04 01:53:23.855603 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.855414    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="af11b6f0-9314-4cb7-ac0d-76ecf4bb665d" containerName="payload-pre-processing"
Nov 04 01:53:23.860395 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.860373    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:23.965781 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.965751    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"]
Nov 04 01:53:23.994766 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.994730    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:23.994959 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.994874    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:23.995033 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.994964    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqw8h\" (UniqueName: \"kubernetes.io/projected/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-kube-api-access-fqw8h\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:23.995095 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.995053    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:23.995154 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.995138    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:23.995241 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.995227    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:23.995327 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.995306    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:23.995480 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.995341    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:23.995480 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:23.995401    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.097038 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.097006    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.097244 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.097056    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.097244 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.097096    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.097244 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.097149    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.097422 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.097242    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.097527 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.097434    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqw8h\" (UniqueName: \"kubernetes.io/projected/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-kube-api-access-fqw8h\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.097590 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.097540    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.097590 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.097558    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.097590 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.097566    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.097756 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.097716    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.097809 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.097795    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.097865 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.097807    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.098072 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.098051    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.098189 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.098166    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.099638 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.099611    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.099957 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.099938    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.158194 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.158164    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqw8h\" (UniqueName: \"kubernetes.io/projected/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-kube-api-access-fqw8h\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.158346 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.158173    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-jjhm5\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.186646 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.186615    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:24.698579 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.698549    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"]
Nov 04 01:53:24.699268 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:53:24.699239    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7e923eb_3db7_4f1d_a9de_fa4a20620bfd.slice/crio-de5db28a8c7e96faf852602670391e91fd4d3bdabf429508e0db815191e0f325 WatchSource:0}: Error finding container de5db28a8c7e96faf852602670391e91fd4d3bdabf429508e0db815191e0f325: Status 404 returned error can't find the container with id de5db28a8c7e96faf852602670391e91fd4d3bdabf429508e0db815191e0f325
Nov 04 01:53:24.701937 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.701906    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:53:24.702020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.701974    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:53:24.702020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:24.702002    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:53:25.374366 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:25.374318    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5" event={"ID":"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd","Type":"ContainerStarted","Data":"87cd9d4f681210cb3f0c4636c5bd2a400341aac763374c98977023d3ec084454"}
Nov 04 01:53:25.374848 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:25.374396    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5" event={"ID":"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd","Type":"ContainerStarted","Data":"de5db28a8c7e96faf852602670391e91fd4d3bdabf429508e0db815191e0f325"}
Nov 04 01:53:25.496993 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:25.496943    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5" podStartSLOduration=2.496925943 podStartE2EDuration="2.496925943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:53:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:53:25.493479948 +0000 UTC m=+1090.614757478" watchObservedRunningTime="2026-11-04 01:53:25.496925943 +0000 UTC m=+1090.618203470"
Nov 04 01:53:26.187009 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:26.186970    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:26.188381 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:26.188361    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-jjhm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 01:53:26.188474 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:26.188409    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5" podUID="a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 01:53:27.187708 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:27.187672    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-jjhm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 01:53:27.188116 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:27.187727    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5" podUID="a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 01:53:28.084264 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:28.084226    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"]
Nov 04 01:53:28.084560 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:28.084532    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh" podUID="70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" containerName="istio-proxy" containerID="cri-o://bd15940d0c0d46fd7a6b14de033e5563ad8069a3802aabbf7fd164d9b0f4654c" gracePeriod=30
Nov 04 01:53:28.091333 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:28.091308    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"]
Nov 04 01:53:28.091555 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:28.091534    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2" podUID="006f9c41-8228-4a5e-9a98-3afb19a40cbc" containerName="istio-proxy" containerID="cri-o://2e2e9bbe2e93b55254877f94981f6bbe1a1778044783326ef2d90ca751813bb4" gracePeriod=30
Nov 04 01:53:28.187303 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:28.187268    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-jjhm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 01:53:28.208607 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:28.207557    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5" podUID="a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 01:53:29.187845 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:29.187802    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-jjhm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 01:53:29.188036 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:29.187872    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5" podUID="a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 01:53:30.187888 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:30.187855    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-jjhm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 01:53:30.188351 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:30.187920    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5" podUID="a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 01:53:30.232166 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:30.232126    6116 patch_prober.go:29] interesting pod/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:53:30.232338 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:30.232207    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2" podUID="006f9c41-8228-4a5e-9a98-3afb19a40cbc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:53:31.187462 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:31.187403    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-jjhm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 01:53:31.187658 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:31.187503    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5" podUID="a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 01:53:32.187426 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:32.187392    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-jjhm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 04 01:53:32.187834 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:32.187486    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5" podUID="a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 04 01:53:33.250273 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.250239    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:33.251141 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.251120    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:33.255590 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.255558    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:53:33.397496 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.397470    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:53:33.439271 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.439244    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:53:33.443055 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.443028    6116 generic.go:356] "Generic (PLEG): container finished" podID="70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" containerID="bd15940d0c0d46fd7a6b14de033e5563ad8069a3802aabbf7fd164d9b0f4654c" exitCode=0
Nov 04 01:53:33.443187 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.443073    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"
Nov 04 01:53:33.443187 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.443108    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh" event={"ID":"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054","Type":"ContainerDied","Data":"bd15940d0c0d46fd7a6b14de033e5563ad8069a3802aabbf7fd164d9b0f4654c"}
Nov 04 01:53:33.443187 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.443162    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh" event={"ID":"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054","Type":"ContainerDied","Data":"b9d0961b097e6df1a98bb417d6dcf44c0f77061c409c6b2111baae9dcae5e02b"}
Nov 04 01:53:33.443354 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.443192    6116 scope.go:122] "RemoveContainer" containerID="bd15940d0c0d46fd7a6b14de033e5563ad8069a3802aabbf7fd164d9b0f4654c"
Nov 04 01:53:33.444774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.444753    6116 generic.go:356] "Generic (PLEG): container finished" podID="006f9c41-8228-4a5e-9a98-3afb19a40cbc" containerID="2e2e9bbe2e93b55254877f94981f6bbe1a1778044783326ef2d90ca751813bb4" exitCode=0
Nov 04 01:53:33.444873 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.444803    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"
Nov 04 01:53:33.444873 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.444815    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2" event={"ID":"006f9c41-8228-4a5e-9a98-3afb19a40cbc","Type":"ContainerDied","Data":"2e2e9bbe2e93b55254877f94981f6bbe1a1778044783326ef2d90ca751813bb4"}
Nov 04 01:53:33.444873 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.444859    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2" event={"ID":"006f9c41-8228-4a5e-9a98-3afb19a40cbc","Type":"ContainerDied","Data":"f2e5a10f549cafae69dcf592edd79aa8db6af2e7d2c35df9bca73a272980fbbb"}
Nov 04 01:53:33.452639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.452619    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-data\") pod \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") "
Nov 04 01:53:33.452738 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.452652    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-token\" (UniqueName: \"kubernetes.io/projected/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-token\") pod \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") "
Nov 04 01:53:33.452738 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.452706    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-data\") pod \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") "
Nov 04 01:53:33.452838 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.452746    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istiod-ca-cert\") pod \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") "
Nov 04 01:53:33.452838 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.452777    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-envoy\") pod \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") "
Nov 04 01:53:33.452838 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.452815    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-envoy\") pod \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") "
Nov 04 01:53:33.452983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.452855    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-credential-socket\") pod \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") "
Nov 04 01:53:33.452983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.452861    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-data" pod "70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" (UID: "70fa17e3-fb7b-4ac0-b7ba-ae560f39f054"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:33.452983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.452937    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istiod-ca-cert\") pod \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") "
Nov 04 01:53:33.453132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.453014    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-podinfo\") pod \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") "
Nov 04 01:53:33.453132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.453018    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-data" pod "006f9c41-8228-4a5e-9a98-3afb19a40cbc" (UID: "006f9c41-8228-4a5e-9a98-3afb19a40cbc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:33.453132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.453050    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-workload-socket\") pod \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") "
Nov 04 01:53:33.453132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.453079    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/006f9c41-8228-4a5e-9a98-3afb19a40cbc-kube-api-access-5s8dd\" (UniqueName: \"kubernetes.io/projected/006f9c41-8228-4a5e-9a98-3afb19a40cbc-kube-api-access-5s8dd\") pod \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") "
Nov 04 01:53:33.453132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.453124    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-workload-socket\") pod \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") "
Nov 04 01:53:33.453373 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.453182    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-podinfo\") pod \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") "
Nov 04 01:53:33.453373 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.453233    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istiod-ca-cert" pod "70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" (UID: "70fa17e3-fb7b-4ac0-b7ba-ae560f39f054"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:53:33.453373 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.453260    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-credential-socket\") pod \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") "
Nov 04 01:53:33.453373 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.453316    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-kube-api-access-jnckc\" (UniqueName: \"kubernetes.io/projected/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-kube-api-access-jnckc\") pod \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") "
Nov 04 01:53:33.453373 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.453352    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-workload-certs\") pod \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") "
Nov 04 01:53:33.453660 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.453381    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-workload-certs\") pod \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\" (UID: \"70fa17e3-fb7b-4ac0-b7ba-ae560f39f054\") "
Nov 04 01:53:33.453660 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.453392    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istiod-ca-cert" pod "006f9c41-8228-4a5e-9a98-3afb19a40cbc" (UID: "006f9c41-8228-4a5e-9a98-3afb19a40cbc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:53:33.453660 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.453414    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-token\" (UniqueName: \"kubernetes.io/projected/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-token\") pod \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\" (UID: \"006f9c41-8228-4a5e-9a98-3afb19a40cbc\") "
Nov 04 01:53:33.453660 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.453607    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-credential-socket" pod "006f9c41-8228-4a5e-9a98-3afb19a40cbc" (UID: "006f9c41-8228-4a5e-9a98-3afb19a40cbc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:33.453660 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.453652    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-credential-socket" pod "70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" (UID: "70fa17e3-fb7b-4ac0-b7ba-ae560f39f054"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:33.454069 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.453547    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-workload-socket" pod "70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" (UID: "70fa17e3-fb7b-4ac0-b7ba-ae560f39f054"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:33.454069 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.453960    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-workload-certs" pod "70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" (UID: "70fa17e3-fb7b-4ac0-b7ba-ae560f39f054"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:33.454181 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.454096    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.454181 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.454115    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.454181 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.454129    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.454181 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.454142    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.454181 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.454157    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.454437 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.454184    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.454437 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.454226    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-workload-certs" pod "006f9c41-8228-4a5e-9a98-3afb19a40cbc" (UID: "006f9c41-8228-4a5e-9a98-3afb19a40cbc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:33.454437 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.454296    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.454437 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.454355    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.454676 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.454602    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-workload-socket" pod "006f9c41-8228-4a5e-9a98-3afb19a40cbc" (UID: "006f9c41-8228-4a5e-9a98-3afb19a40cbc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:33.455354 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.455333    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-token" pod "70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" (UID: "70fa17e3-fb7b-4ac0-b7ba-ae560f39f054"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:53:33.455478 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.455433    6116 scope.go:122] "RemoveContainer" containerID="bd15940d0c0d46fd7a6b14de033e5563ad8069a3802aabbf7fd164d9b0f4654c"
Nov 04 01:53:33.455808 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:33.455779    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd15940d0c0d46fd7a6b14de033e5563ad8069a3802aabbf7fd164d9b0f4654c\": container with ID starting with bd15940d0c0d46fd7a6b14de033e5563ad8069a3802aabbf7fd164d9b0f4654c not found: ID does not exist" containerID="bd15940d0c0d46fd7a6b14de033e5563ad8069a3802aabbf7fd164d9b0f4654c"
Nov 04 01:53:33.455905 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.455820    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd15940d0c0d46fd7a6b14de033e5563ad8069a3802aabbf7fd164d9b0f4654c"} err="failed to get container status \"bd15940d0c0d46fd7a6b14de033e5563ad8069a3802aabbf7fd164d9b0f4654c\": rpc error: code = NotFound desc = could not find container \"bd15940d0c0d46fd7a6b14de033e5563ad8069a3802aabbf7fd164d9b0f4654c\": container with ID starting with bd15940d0c0d46fd7a6b14de033e5563ad8069a3802aabbf7fd164d9b0f4654c not found: ID does not exist"
Nov 04 01:53:33.455905 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.455846    6116 scope.go:122] "RemoveContainer" containerID="2e2e9bbe2e93b55254877f94981f6bbe1a1778044783326ef2d90ca751813bb4"
Nov 04 01:53:33.456023 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.455975    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-envoy" pod "70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" (UID: "70fa17e3-fb7b-4ac0-b7ba-ae560f39f054"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:33.456023 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.455985    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-envoy" pod "006f9c41-8228-4a5e-9a98-3afb19a40cbc" (UID: "006f9c41-8228-4a5e-9a98-3afb19a40cbc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:33.456023 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.455995    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/006f9c41-8228-4a5e-9a98-3afb19a40cbc-kube-api-access-5s8dd" pod "006f9c41-8228-4a5e-9a98-3afb19a40cbc" (UID: "006f9c41-8228-4a5e-9a98-3afb19a40cbc"). InnerVolumeSpecName "kube-api-access-5s8dd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:53:33.456415 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.456384    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-podinfo" pod "006f9c41-8228-4a5e-9a98-3afb19a40cbc" (UID: "006f9c41-8228-4a5e-9a98-3afb19a40cbc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:53:33.456895 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.456843    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-podinfo" pod "70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" (UID: "70fa17e3-fb7b-4ac0-b7ba-ae560f39f054"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:53:33.457268 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.457250    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-kube-api-access-jnckc" pod "70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" (UID: "70fa17e3-fb7b-4ac0-b7ba-ae560f39f054"). InnerVolumeSpecName "kube-api-access-jnckc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:53:33.457320 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.457259    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-token" pod "006f9c41-8228-4a5e-9a98-3afb19a40cbc" (UID: "006f9c41-8228-4a5e-9a98-3afb19a40cbc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:53:33.465224 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.465201    6116 scope.go:122] "RemoveContainer" containerID="35ad9202b4c9a79236b9f41841dee7b5229eface16f6ba95a809554dce2b1fb7"
Nov 04 01:53:33.474335 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.474314    6116 scope.go:122] "RemoveContainer" containerID="2e2e9bbe2e93b55254877f94981f6bbe1a1778044783326ef2d90ca751813bb4"
Nov 04 01:53:33.474647 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:33.474622    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e2e9bbe2e93b55254877f94981f6bbe1a1778044783326ef2d90ca751813bb4\": container with ID starting with 2e2e9bbe2e93b55254877f94981f6bbe1a1778044783326ef2d90ca751813bb4 not found: ID does not exist" containerID="2e2e9bbe2e93b55254877f94981f6bbe1a1778044783326ef2d90ca751813bb4"
Nov 04 01:53:33.474737 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.474661    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e2e9bbe2e93b55254877f94981f6bbe1a1778044783326ef2d90ca751813bb4"} err="failed to get container status \"2e2e9bbe2e93b55254877f94981f6bbe1a1778044783326ef2d90ca751813bb4\": rpc error: code = NotFound desc = could not find container \"2e2e9bbe2e93b55254877f94981f6bbe1a1778044783326ef2d90ca751813bb4\": container with ID starting with 2e2e9bbe2e93b55254877f94981f6bbe1a1778044783326ef2d90ca751813bb4 not found: ID does not exist"
Nov 04 01:53:33.474737 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.474680    6116 scope.go:122] "RemoveContainer" containerID="35ad9202b4c9a79236b9f41841dee7b5229eface16f6ba95a809554dce2b1fb7"
Nov 04 01:53:33.474921 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:33.474904    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35ad9202b4c9a79236b9f41841dee7b5229eface16f6ba95a809554dce2b1fb7\": container with ID starting with 35ad9202b4c9a79236b9f41841dee7b5229eface16f6ba95a809554dce2b1fb7 not found: ID does not exist" containerID="35ad9202b4c9a79236b9f41841dee7b5229eface16f6ba95a809554dce2b1fb7"
Nov 04 01:53:33.474960 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.474928    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35ad9202b4c9a79236b9f41841dee7b5229eface16f6ba95a809554dce2b1fb7"} err="failed to get container status \"35ad9202b4c9a79236b9f41841dee7b5229eface16f6ba95a809554dce2b1fb7\": rpc error: code = NotFound desc = could not find container \"35ad9202b4c9a79236b9f41841dee7b5229eface16f6ba95a809554dce2b1fb7\": container with ID starting with 35ad9202b4c9a79236b9f41841dee7b5229eface16f6ba95a809554dce2b1fb7 not found: ID does not exist"
Nov 04 01:53:33.559750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.559680    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnckc\" (UniqueName: \"kubernetes.io/projected/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-kube-api-access-jnckc\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.559750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.559710    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.559750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.559724    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.559750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.559736    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.559750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.559748    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.559750 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.559759    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.560073 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.559771    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/006f9c41-8228-4a5e-9a98-3afb19a40cbc-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.560073 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.559783    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5s8dd\" (UniqueName: \"kubernetes.io/projected/006f9c41-8228-4a5e-9a98-3afb19a40cbc-kube-api-access-5s8dd\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.560073 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.559794    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/006f9c41-8228-4a5e-9a98-3afb19a40cbc-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.560073 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.559805    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:33.950386 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.950345    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"]
Nov 04 01:53:33.955117 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:33.955090    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh"]
Nov 04 01:53:34.063890 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:34.063795    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"]
Nov 04 01:53:34.068083 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:34.068054    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2"]
Nov 04 01:53:35.028391 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:35.028360    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="006f9c41-8228-4a5e-9a98-3afb19a40cbc" path="/var/lib/kubelet/pods/006f9c41-8228-4a5e-9a98-3afb19a40cbc/volumes"
Nov 04 01:53:35.028887 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:35.028870    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" path="/var/lib/kubelet/pods/70fa17e3-fb7b-4ac0-b7ba-ae560f39f054/volumes"
Nov 04 01:53:37.829786 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:37.829758    6116 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-84cffd7d1248: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:37.829786 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:37.829783    6116 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:37.830196 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:37.829841    6116 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials podName:233cb7a1-2c92-4544-afe5-5aae953288e1 nodeName:}" failed. No retries permitted until 2026-11-04 01:54:09.829828444 +0000 UTC m=+1134.951105949 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials") pod "payload-processing-external-model-5497dccdcc-bbs4k" (UID: "233cb7a1-2c92-4544-afe5-5aae953288e1") : secret "agc-cred-84cffd7d1248" not found
Nov 04 01:53:41.992164 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:41.992126    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:53:49.034015 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.033976    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx"]
Nov 04 01:53:49.034755 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.034734    6116 cpu_manager.go:414] "containerMap: removing container" podUID="006f9c41-8228-4a5e-9a98-3afb19a40cbc" containerName="istio-proxy"
Nov 04 01:53:49.034755 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.034755    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="006f9c41-8228-4a5e-9a98-3afb19a40cbc" containerName="istio-proxy"
Nov 04 01:53:49.034917 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.034777    6116 cpu_manager.go:414] "containerMap: removing container" podUID="70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" containerName="istio-proxy"
Nov 04 01:53:49.034917 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.034783    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" containerName="istio-proxy"
Nov 04 01:53:49.034917 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.034816    6116 cpu_manager.go:414] "containerMap: removing container" podUID="006f9c41-8228-4a5e-9a98-3afb19a40cbc" containerName="istio-proxy"
Nov 04 01:53:49.034917 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.034823    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="006f9c41-8228-4a5e-9a98-3afb19a40cbc" containerName="istio-proxy"
Nov 04 01:53:49.035092 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.034960    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="006f9c41-8228-4a5e-9a98-3afb19a40cbc" containerName="istio-proxy"
Nov 04 01:53:49.035092 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.034978    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="006f9c41-8228-4a5e-9a98-3afb19a40cbc" containerName="istio-proxy"
Nov 04 01:53:49.035092 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.034990    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="70fa17e3-fb7b-4ac0-b7ba-ae560f39f054" containerName="istio-proxy"
Nov 04 01:53:49.039862 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.039841    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx"
Nov 04 01:53:49.055133 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.055110    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fglps\" (UniqueName: \"kubernetes.io/projected/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-kube-api-access-fglps\") pod \"maas-api-revoke-sub-5487d7d8-s98rx\" (UID: \"2fb79b6e-5c0e-491b-ae5b-337ecd99901e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx"
Nov 04 01:53:49.055240 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.055150    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5487d7d8-s98rx\" (UID: \"2fb79b6e-5c0e-491b-ae5b-337ecd99901e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx"
Nov 04 01:53:49.055363 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.055350    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5487d7d8-s98rx\" (UID: \"2fb79b6e-5c0e-491b-ae5b-337ecd99901e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx"
Nov 04 01:53:49.095887 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.095848    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx"]
Nov 04 01:53:49.116949 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.116916    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 01:53:49.141491 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.141421    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bj8b6\""
Nov 04 01:53:49.141491 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.141421    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:53:49.141491 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.141421    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:53:49.156801 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.156765    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fglps\" (UniqueName: \"kubernetes.io/projected/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-kube-api-access-fglps\") pod \"maas-api-revoke-sub-5487d7d8-s98rx\" (UID: \"2fb79b6e-5c0e-491b-ae5b-337ecd99901e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx"
Nov 04 01:53:49.156956 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.156826    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5487d7d8-s98rx\" (UID: \"2fb79b6e-5c0e-491b-ae5b-337ecd99901e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx"
Nov 04 01:53:49.156956 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.156908    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5487d7d8-s98rx\" (UID: \"2fb79b6e-5c0e-491b-ae5b-337ecd99901e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx"
Nov 04 01:53:49.157442 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.157423    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5487d7d8-s98rx\" (UID: \"2fb79b6e-5c0e-491b-ae5b-337ecd99901e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx"
Nov 04 01:53:49.159616 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.159591    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5487d7d8-s98rx\" (UID: \"2fb79b6e-5c0e-491b-ae5b-337ecd99901e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx"
Nov 04 01:53:49.179761 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.179730    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"]
Nov 04 01:53:49.179961 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.179941    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz" podUID="4229f224-e59f-4caa-9837-f303c79c72a2" containerName="istio-proxy" containerID="cri-o://6575347f728cf6f78a12bb0c44e1c6459b10370f3c76a2e15d3d0d0717c1e1b1" gracePeriod=30
Nov 04 01:53:49.217836 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.217805    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fglps\" (UniqueName: \"kubernetes.io/projected/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-kube-api-access-fglps\") pod \"maas-api-revoke-sub-5487d7d8-s98rx\" (UID: \"2fb79b6e-5c0e-491b-ae5b-337ecd99901e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx"
Nov 04 01:53:49.353377 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.353283    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx"
Nov 04 01:53:49.730733 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:49.730710    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx"]
Nov 04 01:53:49.732727 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:53:49.732702    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fb79b6e_5c0e_491b_ae5b_337ecd99901e.slice/crio-156140fd64035f8f8dc25cab234cfc891f6d511fd4176034cd815be213ee92e4 WatchSource:0}: Error finding container 156140fd64035f8f8dc25cab234cfc891f6d511fd4176034cd815be213ee92e4: Status 404 returned error can't find the container with id 156140fd64035f8f8dc25cab234cfc891f6d511fd4176034cd815be213ee92e4
Nov 04 01:53:50.606485 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:50.606422    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx" event={"ID":"2fb79b6e-5c0e-491b-ae5b-337ecd99901e","Type":"ContainerStarted","Data":"62205101a5b80c7bc68a323c57017a6dadf6cb8e8e734567ccf94d44666c9fc6"}
Nov 04 01:53:50.606914 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:50.606543    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx" event={"ID":"2fb79b6e-5c0e-491b-ae5b-337ecd99901e","Type":"ContainerStarted","Data":"156140fd64035f8f8dc25cab234cfc891f6d511fd4176034cd815be213ee92e4"}
Nov 04 01:53:50.723600 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:50.723550    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx" podStartSLOduration=2.723536636 podStartE2EDuration="2.723536636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:53:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:53:50.721172167 +0000 UTC m=+1115.842449693" watchObservedRunningTime="2026-11-04 01:53:50.723536636 +0000 UTC m=+1115.844814161"
Nov 04 01:53:51.614707 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:51.614672    6116 generic.go:356] "Generic (PLEG): container finished" podID="2fb79b6e-5c0e-491b-ae5b-337ecd99901e" containerID="62205101a5b80c7bc68a323c57017a6dadf6cb8e8e734567ccf94d44666c9fc6" exitCode=0
Nov 04 01:53:51.615157 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:51.614756    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx" event={"ID":"2fb79b6e-5c0e-491b-ae5b-337ecd99901e","Type":"ContainerDied","Data":"62205101a5b80c7bc68a323c57017a6dadf6cb8e8e734567ccf94d44666c9fc6"}
Nov 04 01:53:52.766718 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:52.766689    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx"
Nov 04 01:53:52.805613 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:52.805582    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-kube-api-access-fglps\" (UniqueName: \"kubernetes.io/projected/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-kube-api-access-fglps\") pod \"2fb79b6e-5c0e-491b-ae5b-337ecd99901e\" (UID: \"2fb79b6e-5c0e-491b-ae5b-337ecd99901e\") "
Nov 04 01:53:52.805821 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:52.805760    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-maas-api-openshift-ca\") pod \"2fb79b6e-5c0e-491b-ae5b-337ecd99901e\" (UID: \"2fb79b6e-5c0e-491b-ae5b-337ecd99901e\") "
Nov 04 01:53:52.805821 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:52.805781    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-maas-api-xks-ca\") pod \"2fb79b6e-5c0e-491b-ae5b-337ecd99901e\" (UID: \"2fb79b6e-5c0e-491b-ae5b-337ecd99901e\") "
Nov 04 01:53:52.806157 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:52.806130    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-maas-api-openshift-ca" pod "2fb79b6e-5c0e-491b-ae5b-337ecd99901e" (UID: "2fb79b6e-5c0e-491b-ae5b-337ecd99901e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:53:52.807903 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:52.807881    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-kube-api-access-fglps" pod "2fb79b6e-5c0e-491b-ae5b-337ecd99901e" (UID: "2fb79b6e-5c0e-491b-ae5b-337ecd99901e"). InnerVolumeSpecName "kube-api-access-fglps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:53:52.807903 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:52.807890    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-maas-api-xks-ca" pod "2fb79b6e-5c0e-491b-ae5b-337ecd99901e" (UID: "2fb79b6e-5c0e-491b-ae5b-337ecd99901e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:53:52.907923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:52.907894    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-maas-api-openshift-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:52.907923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:52.907920    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-maas-api-xks-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:52.907923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:52.907930    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fglps\" (UniqueName: \"kubernetes.io/projected/2fb79b6e-5c0e-491b-ae5b-337ecd99901e-kube-api-access-fglps\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:53.629495 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:53.629441    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx"
Nov 04 01:53:53.629495 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:53.629476    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx" event={"ID":"2fb79b6e-5c0e-491b-ae5b-337ecd99901e","Type":"ContainerDied","Data":"156140fd64035f8f8dc25cab234cfc891f6d511fd4176034cd815be213ee92e4"}
Nov 04 01:53:53.629699 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:53.629525    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="156140fd64035f8f8dc25cab234cfc891f6d511fd4176034cd815be213ee92e4"
Nov 04 01:53:54.429830 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.429805    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:53:54.527963 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.527869    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4229f224-e59f-4caa-9837-f303c79c72a2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4229f224-e59f-4caa-9837-f303c79c72a2-istio-podinfo\") pod \"4229f224-e59f-4caa-9837-f303c79c72a2\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") "
Nov 04 01:53:54.527963 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.527927    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4229f224-e59f-4caa-9837-f303c79c72a2-istio-token\" (UniqueName: \"kubernetes.io/projected/4229f224-e59f-4caa-9837-f303c79c72a2-istio-token\") pod \"4229f224-e59f-4caa-9837-f303c79c72a2\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") "
Nov 04 01:53:54.528189 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.528027    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-credential-socket\") pod \"4229f224-e59f-4caa-9837-f303c79c72a2\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") "
Nov 04 01:53:54.528189 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.528136    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4229f224-e59f-4caa-9837-f303c79c72a2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4229f224-e59f-4caa-9837-f303c79c72a2-istiod-ca-cert\") pod \"4229f224-e59f-4caa-9837-f303c79c72a2\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") "
Nov 04 01:53:54.528291 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.528221    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4229f224-e59f-4caa-9837-f303c79c72a2-kube-api-access-zhlc9\" (UniqueName: \"kubernetes.io/projected/4229f224-e59f-4caa-9837-f303c79c72a2-kube-api-access-zhlc9\") pod \"4229f224-e59f-4caa-9837-f303c79c72a2\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") "
Nov 04 01:53:54.528343 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.528298    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-credential-socket" pod "4229f224-e59f-4caa-9837-f303c79c72a2" (UID: "4229f224-e59f-4caa-9837-f303c79c72a2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:54.528343 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.528320    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-workload-socket\") pod \"4229f224-e59f-4caa-9837-f303c79c72a2\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") "
Nov 04 01:53:54.528441 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.528351    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-istio-envoy\") pod \"4229f224-e59f-4caa-9837-f303c79c72a2\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") "
Nov 04 01:53:54.528441 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.528420    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-workload-certs\") pod \"4229f224-e59f-4caa-9837-f303c79c72a2\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") "
Nov 04 01:53:54.528601 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.528486    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-istio-data\") pod \"4229f224-e59f-4caa-9837-f303c79c72a2\" (UID: \"4229f224-e59f-4caa-9837-f303c79c72a2\") "
Nov 04 01:53:54.529053 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.529023    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-workload-socket" pod "4229f224-e59f-4caa-9837-f303c79c72a2" (UID: "4229f224-e59f-4caa-9837-f303c79c72a2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:54.529166 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.529065    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:54.529292 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.529269    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-workload-certs" pod "4229f224-e59f-4caa-9837-f303c79c72a2" (UID: "4229f224-e59f-4caa-9837-f303c79c72a2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:54.529355 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.529300    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-istio-data" pod "4229f224-e59f-4caa-9837-f303c79c72a2" (UID: "4229f224-e59f-4caa-9837-f303c79c72a2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:54.529355 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.529333    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4229f224-e59f-4caa-9837-f303c79c72a2-istiod-ca-cert" pod "4229f224-e59f-4caa-9837-f303c79c72a2" (UID: "4229f224-e59f-4caa-9837-f303c79c72a2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:53:54.530408 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.530375    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4229f224-e59f-4caa-9837-f303c79c72a2-istio-podinfo" pod "4229f224-e59f-4caa-9837-f303c79c72a2" (UID: "4229f224-e59f-4caa-9837-f303c79c72a2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:53:54.530668 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.530638    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4229f224-e59f-4caa-9837-f303c79c72a2-istio-token" pod "4229f224-e59f-4caa-9837-f303c79c72a2" (UID: "4229f224-e59f-4caa-9837-f303c79c72a2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:53:54.530764 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.530741    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4229f224-e59f-4caa-9837-f303c79c72a2-kube-api-access-zhlc9" pod "4229f224-e59f-4caa-9837-f303c79c72a2" (UID: "4229f224-e59f-4caa-9837-f303c79c72a2"). InnerVolumeSpecName "kube-api-access-zhlc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:53:54.531214 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.531192    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-istio-envoy" pod "4229f224-e59f-4caa-9837-f303c79c72a2" (UID: "4229f224-e59f-4caa-9837-f303c79c72a2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:53:54.630793 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.630742    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4229f224-e59f-4caa-9837-f303c79c72a2-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:54.630793 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.630790    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhlc9\" (UniqueName: \"kubernetes.io/projected/4229f224-e59f-4caa-9837-f303c79c72a2-kube-api-access-zhlc9\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:54.631136 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.630805    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:54.631136 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.630817    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:54.631136 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.630830    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:54.631136 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.630844    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4229f224-e59f-4caa-9837-f303c79c72a2-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:54.631136 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.630857    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4229f224-e59f-4caa-9837-f303c79c72a2-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:54.631136 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.630869    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4229f224-e59f-4caa-9837-f303c79c72a2-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:53:54.650348 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.650314    6116 generic.go:356] "Generic (PLEG): container finished" podID="4229f224-e59f-4caa-9837-f303c79c72a2" containerID="6575347f728cf6f78a12bb0c44e1c6459b10370f3c76a2e15d3d0d0717c1e1b1" exitCode=0
Nov 04 01:53:54.650552 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.650375    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"
Nov 04 01:53:54.650552 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.650390    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz" event={"ID":"4229f224-e59f-4caa-9837-f303c79c72a2","Type":"ContainerDied","Data":"6575347f728cf6f78a12bb0c44e1c6459b10370f3c76a2e15d3d0d0717c1e1b1"}
Nov 04 01:53:54.650552 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.650489    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz" event={"ID":"4229f224-e59f-4caa-9837-f303c79c72a2","Type":"ContainerDied","Data":"822754703b7438159664fdae801724b47db2d92a6f0f329f204bf01af4b0a198"}
Nov 04 01:53:54.650552 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.650512    6116 scope.go:122] "RemoveContainer" containerID="6575347f728cf6f78a12bb0c44e1c6459b10370f3c76a2e15d3d0d0717c1e1b1"
Nov 04 01:53:54.661390 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.661371    6116 scope.go:122] "RemoveContainer" containerID="6575347f728cf6f78a12bb0c44e1c6459b10370f3c76a2e15d3d0d0717c1e1b1"
Nov 04 01:53:54.661686 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:53:54.661665    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6575347f728cf6f78a12bb0c44e1c6459b10370f3c76a2e15d3d0d0717c1e1b1\": container with ID starting with 6575347f728cf6f78a12bb0c44e1c6459b10370f3c76a2e15d3d0d0717c1e1b1 not found: ID does not exist" containerID="6575347f728cf6f78a12bb0c44e1c6459b10370f3c76a2e15d3d0d0717c1e1b1"
Nov 04 01:53:54.661734 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.661694    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6575347f728cf6f78a12bb0c44e1c6459b10370f3c76a2e15d3d0d0717c1e1b1"} err="failed to get container status \"6575347f728cf6f78a12bb0c44e1c6459b10370f3c76a2e15d3d0d0717c1e1b1\": rpc error: code = NotFound desc = could not find container \"6575347f728cf6f78a12bb0c44e1c6459b10370f3c76a2e15d3d0d0717c1e1b1\": container with ID starting with 6575347f728cf6f78a12bb0c44e1c6459b10370f3c76a2e15d3d0d0717c1e1b1 not found: ID does not exist"
Nov 04 01:53:54.818498 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.818401    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"]
Nov 04 01:53:54.823114 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:54.823088    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz"]
Nov 04 01:53:55.029121 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:55.029085    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4229f224-e59f-4caa-9837-f303c79c72a2" path="/var/lib/kubelet/pods/4229f224-e59f-4caa-9837-f303c79c72a2/volumes"
Nov 04 01:53:57.281113 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:53:57.281082    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:54:09.923260 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:54:09.923222    6116 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-84cffd7d1248: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:54:09.923260 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:54:09.923251    6116 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:54:09.923760 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:54:09.923314    6116 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials podName:233cb7a1-2c92-4544-afe5-5aae953288e1 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:13.92329912 +0000 UTC m=+1199.044576624 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials") pod "payload-processing-external-model-5497dccdcc-bbs4k" (UID: "233cb7a1-2c92-4544-afe5-5aae953288e1") : secret "agc-cred-84cffd7d1248" not found
Nov 04 01:54:11.338477 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.338427    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"]
Nov 04 01:54:11.339013 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.338998    6116 cpu_manager.go:414] "containerMap: removing container" podUID="4229f224-e59f-4caa-9837-f303c79c72a2" containerName="istio-proxy"
Nov 04 01:54:11.339060 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.339015    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4229f224-e59f-4caa-9837-f303c79c72a2" containerName="istio-proxy"
Nov 04 01:54:11.339060 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.339045    6116 cpu_manager.go:414] "containerMap: removing container" podUID="2fb79b6e-5c0e-491b-ae5b-337ecd99901e" containerName="revoke-keys"
Nov 04 01:54:11.339060 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.339050    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2fb79b6e-5c0e-491b-ae5b-337ecd99901e" containerName="revoke-keys"
Nov 04 01:54:11.339151 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.339144    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="4229f224-e59f-4caa-9837-f303c79c72a2" containerName="istio-proxy"
Nov 04 01:54:11.339180 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.339157    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="2fb79b6e-5c0e-491b-ae5b-337ecd99901e" containerName="revoke-keys"
Nov 04 01:54:11.342531 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.342506    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"
Nov 04 01:54:11.445552 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.445519    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c0ab5f\"/\"kube-root-ca.crt\""
Nov 04 01:54:11.445552 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.445519    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c0ab5f\"/\"openshift-service-ca.crt\""
Nov 04 01:54:11.445552 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.445519    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c0ab5f\"/\"payload-processing-external-model-plugins-e2e-extmod-b-c0ab5f\""
Nov 04 01:54:11.445874 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.445533    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-c0ab5f\"/\"agc-cred-b7f631c0cffe\""
Nov 04 01:54:11.445874 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.445615    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-c0ab5f\"/\"payload-processing-external-model-e2e--bebee751-dockercfg-7bfjn\""
Nov 04 01:54:11.445874 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.445726    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c0ab5f\"/\"routing-overlay\""
Nov 04 01:54:11.446559 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.446540    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tblrn\" (UniqueName: \"kubernetes.io/projected/fee5848c-8694-4d7e-96d9-7536e13022a3-kube-api-access-tblrn\") pod \"payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn\" (UID: \"fee5848c-8694-4d7e-96d9-7536e13022a3\") " pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"
Nov 04 01:54:11.446642 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.446604    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/fee5848c-8694-4d7e-96d9-7536e13022a3-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn\" (UID: \"fee5848c-8694-4d7e-96d9-7536e13022a3\") " pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"
Nov 04 01:54:11.446797 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.446777    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/fee5848c-8694-4d7e-96d9-7536e13022a3-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn\" (UID: \"fee5848c-8694-4d7e-96d9-7536e13022a3\") " pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"
Nov 04 01:54:11.446864 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.446846    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fee5848c-8694-4d7e-96d9-7536e13022a3-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn\" (UID: \"fee5848c-8694-4d7e-96d9-7536e13022a3\") " pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"
Nov 04 01:54:11.448695 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.448674    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"]
Nov 04 01:54:11.547932 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.547895    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/fee5848c-8694-4d7e-96d9-7536e13022a3-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn\" (UID: \"fee5848c-8694-4d7e-96d9-7536e13022a3\") " pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"
Nov 04 01:54:11.548112 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.547953    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fee5848c-8694-4d7e-96d9-7536e13022a3-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn\" (UID: \"fee5848c-8694-4d7e-96d9-7536e13022a3\") " pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"
Nov 04 01:54:11.548112 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.548021    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tblrn\" (UniqueName: \"kubernetes.io/projected/fee5848c-8694-4d7e-96d9-7536e13022a3-kube-api-access-tblrn\") pod \"payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn\" (UID: \"fee5848c-8694-4d7e-96d9-7536e13022a3\") " pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"
Nov 04 01:54:11.548112 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.548085    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/fee5848c-8694-4d7e-96d9-7536e13022a3-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn\" (UID: \"fee5848c-8694-4d7e-96d9-7536e13022a3\") " pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"
Nov 04 01:54:11.548577 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.548554    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/fee5848c-8694-4d7e-96d9-7536e13022a3-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn\" (UID: \"fee5848c-8694-4d7e-96d9-7536e13022a3\") " pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"
Nov 04 01:54:11.548784 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.548766    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fee5848c-8694-4d7e-96d9-7536e13022a3-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn\" (UID: \"fee5848c-8694-4d7e-96d9-7536e13022a3\") " pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"
Nov 04 01:54:11.550673 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.550654    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/fee5848c-8694-4d7e-96d9-7536e13022a3-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn\" (UID: \"fee5848c-8694-4d7e-96d9-7536e13022a3\") " pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"
Nov 04 01:54:11.608680 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.608617    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tblrn\" (UniqueName: \"kubernetes.io/projected/fee5848c-8694-4d7e-96d9-7536e13022a3-kube-api-access-tblrn\") pod \"payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn\" (UID: \"fee5848c-8694-4d7e-96d9-7536e13022a3\") " pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"
Nov 04 01:54:11.656887 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:11.656851    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"
Nov 04 01:54:11.992273 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:54:11.992235    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:54:12.041503 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:12.041473    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"]
Nov 04 01:54:12.042547 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:54:12.042510    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfee5848c_8694_4d7e_96d9_7536e13022a3.slice/crio-c8c1158472f36bef7dc8cba642c4ccba5da0aba203ca38731f85ab553146c736 WatchSource:0}: Error finding container c8c1158472f36bef7dc8cba642c4ccba5da0aba203ca38731f85ab553146c736: Status 404 returned error can't find the container with id c8c1158472f36bef7dc8cba642c4ccba5da0aba203ca38731f85ab553146c736
Nov 04 01:54:12.784263 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:12.784223    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn" event={"ID":"fee5848c-8694-4d7e-96d9-7536e13022a3","Type":"ContainerStarted","Data":"277911ca153e731f420a6c8041e9cfe69913be1aa22f03d7cc9867eec5eb61a2"}
Nov 04 01:54:12.784775 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:12.784287    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"
Nov 04 01:54:12.784775 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:12.784304    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn" event={"ID":"fee5848c-8694-4d7e-96d9-7536e13022a3","Type":"ContainerStarted","Data":"c8c1158472f36bef7dc8cba642c4ccba5da0aba203ca38731f85ab553146c736"}
Nov 04 01:54:12.900861 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:12.900812    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn" podStartSLOduration=1.90079682 podStartE2EDuration="1.90079682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:54:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:54:12.899529722 +0000 UTC m=+1138.020807248" watchObservedRunningTime="2026-11-04 01:54:12.90079682 +0000 UTC m=+1138.022074345"
Nov 04 01:54:23.795603 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:23.795576    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"
Nov 04 01:54:29.047225 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.047188    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"]
Nov 04 01:54:29.052955 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.052932    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.084295 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.084252    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-data\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.085499 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.085469    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-envoy\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.089477 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.085730    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a649747c-ccad-4629-8be4-43fc827eb3f9-istiod-ca-cert\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.089710 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.089689    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-credential-socket\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.089881 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.089864    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-token\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.090218 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.090202    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-workload-certs\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.090383 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.090368    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-workload-socket\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.090577 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.090561    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5npq\" (UniqueName: \"kubernetes.io/projected/a649747c-ccad-4629-8be4-43fc827eb3f9-kube-api-access-z5npq\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.090743 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.090729    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-podinfo\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.106562 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.106513    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-27bb49-openshift-default-dockercfg-2g5dr\""
Nov 04 01:54:29.192612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.192578    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a649747c-ccad-4629-8be4-43fc827eb3f9-istiod-ca-cert\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.192612 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.192615    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-credential-socket\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.192819 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.192631    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-token\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.192819 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.192751    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-workload-certs\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.192819 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.192779    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-workload-socket\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.192819 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.192812    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5npq\" (UniqueName: \"kubernetes.io/projected/a649747c-ccad-4629-8be4-43fc827eb3f9-kube-api-access-z5npq\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.193020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.192846    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-podinfo\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.193020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.192912    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-data\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.193020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.192965    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-envoy\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.193536 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.193232    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-workload-certs\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.193536 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.193320    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a649747c-ccad-4629-8be4-43fc827eb3f9-istiod-ca-cert\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.193536 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.193415    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-workload-socket\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.193536 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.193508    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-data\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.193764 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.193656    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-credential-socket\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.195367 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.195347    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-envoy\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.195619 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.195586    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-podinfo\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.254930 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.254616    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-token\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.255132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.255105    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5npq\" (UniqueName: \"kubernetes.io/projected/a649747c-ccad-4629-8be4-43fc827eb3f9-kube-api-access-z5npq\") pod \"e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") " pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.257493 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.257465    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"]
Nov 04 01:54:29.366824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.366741    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:29.799161 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.799128    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"]
Nov 04 01:54:29.800035 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:54:29.800011    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda649747c_ccad_4629_8be4_43fc827eb3f9.slice/crio-d99c56bc2ce5904193baae14afc6631a4d7723627c771202c8c299541baa5d9b WatchSource:0}: Error finding container d99c56bc2ce5904193baae14afc6631a4d7723627c771202c8c299541baa5d9b: Status 404 returned error can't find the container with id d99c56bc2ce5904193baae14afc6631a4d7723627c771202c8c299541baa5d9b
Nov 04 01:54:29.802382 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.802349    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:54:29.802483 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.802420    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:54:29.802527 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.802482    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:54:29.894539 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.894036    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6" event={"ID":"a649747c-ccad-4629-8be4-43fc827eb3f9","Type":"ContainerStarted","Data":"589c975d481f8234f9faf58c83e27b0a5a1f8f3d2dc87189ba1c69ea1cfd5f2a"}
Nov 04 01:54:29.894539 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:29.894094    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6" event={"ID":"a649747c-ccad-4629-8be4-43fc827eb3f9","Type":"ContainerStarted","Data":"d99c56bc2ce5904193baae14afc6631a4d7723627c771202c8c299541baa5d9b"}
Nov 04 01:54:30.016098 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:30.016045    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6" podStartSLOduration=1.016030987 podStartE2EDuration="1.016030987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:54:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:54:30.012902332 +0000 UTC m=+1155.134179860" watchObservedRunningTime="2026-11-04 01:54:30.016030987 +0000 UTC m=+1155.137308512"
Nov 04 01:54:30.367472 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:30.367404    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:30.373187 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:30.373161    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:30.900674 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:30.900641    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:30.901741 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:30.901721    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:54:32.281512 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:32.281480    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:54:41.993144 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:54:41.993105    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:54:54.637036 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:54.637003    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:54:58.491597 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:58.491563    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr"]
Nov 04 01:54:58.496711 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:58.496693    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr"
Nov 04 01:54:58.549947 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:58.549912    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 01:54:58.554671 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:58.554639    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr"]
Nov 04 01:54:58.599367 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:58.599339    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:54:58.599542 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:58.599427    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:54:58.599542 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:58.599434    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bj8b6\""
Nov 04 01:54:58.601032 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:58.601012    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf7d56e8-d90e-4557-b72d-a3ac56015867-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7d68d3e6-b7sjr\" (UID: \"cf7d56e8-d90e-4557-b72d-a3ac56015867\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr"
Nov 04 01:54:58.601163 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:58.601149    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf7d56e8-d90e-4557-b72d-a3ac56015867-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7d68d3e6-b7sjr\" (UID: \"cf7d56e8-d90e-4557-b72d-a3ac56015867\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr"
Nov 04 01:54:58.601202 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:58.601190    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5r5r\" (UniqueName: \"kubernetes.io/projected/cf7d56e8-d90e-4557-b72d-a3ac56015867-kube-api-access-l5r5r\") pod \"maas-api-revoke-sub-7d68d3e6-b7sjr\" (UID: \"cf7d56e8-d90e-4557-b72d-a3ac56015867\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr"
Nov 04 01:54:58.702943 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:58.702904    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf7d56e8-d90e-4557-b72d-a3ac56015867-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7d68d3e6-b7sjr\" (UID: \"cf7d56e8-d90e-4557-b72d-a3ac56015867\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr"
Nov 04 01:54:58.703111 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:58.703043    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf7d56e8-d90e-4557-b72d-a3ac56015867-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7d68d3e6-b7sjr\" (UID: \"cf7d56e8-d90e-4557-b72d-a3ac56015867\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr"
Nov 04 01:54:58.703111 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:58.703099    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5r5r\" (UniqueName: \"kubernetes.io/projected/cf7d56e8-d90e-4557-b72d-a3ac56015867-kube-api-access-l5r5r\") pod \"maas-api-revoke-sub-7d68d3e6-b7sjr\" (UID: \"cf7d56e8-d90e-4557-b72d-a3ac56015867\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr"
Nov 04 01:54:58.703714 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:58.703661    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf7d56e8-d90e-4557-b72d-a3ac56015867-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7d68d3e6-b7sjr\" (UID: \"cf7d56e8-d90e-4557-b72d-a3ac56015867\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr"
Nov 04 01:54:58.705433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:58.705404    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf7d56e8-d90e-4557-b72d-a3ac56015867-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7d68d3e6-b7sjr\" (UID: \"cf7d56e8-d90e-4557-b72d-a3ac56015867\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr"
Nov 04 01:54:58.764328 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:58.764253    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5r5r\" (UniqueName: \"kubernetes.io/projected/cf7d56e8-d90e-4557-b72d-a3ac56015867-kube-api-access-l5r5r\") pod \"maas-api-revoke-sub-7d68d3e6-b7sjr\" (UID: \"cf7d56e8-d90e-4557-b72d-a3ac56015867\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr"
Nov 04 01:54:58.811230 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:58.811199    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr"
Nov 04 01:54:59.196015 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:59.195986    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr"]
Nov 04 01:54:59.196858 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:54:59.196834    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf7d56e8_d90e_4557_b72d_a3ac56015867.slice/crio-d91313cffa01ae3e9d784060c165689f89a16c7b26a8a8319b20f8c9297d5922 WatchSource:0}: Error finding container d91313cffa01ae3e9d784060c165689f89a16c7b26a8a8319b20f8c9297d5922: Status 404 returned error can't find the container with id d91313cffa01ae3e9d784060c165689f89a16c7b26a8a8319b20f8c9297d5922
Nov 04 01:54:59.814171 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:59.814082    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5"]
Nov 04 01:54:59.818274 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:59.818249    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5"
Nov 04 01:54:59.876107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:59.876075    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5"]
Nov 04 01:54:59.920738 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:59.920703    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/843e5f1b-1c67-4b12-b617-1b4f293baa0c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-da8cb04f-nzhc5\" (UID: \"843e5f1b-1c67-4b12-b617-1b4f293baa0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5"
Nov 04 01:54:59.920905 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:59.920848    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj6ls\" (UniqueName: \"kubernetes.io/projected/843e5f1b-1c67-4b12-b617-1b4f293baa0c-kube-api-access-qj6ls\") pod \"maas-api-revoke-sub-da8cb04f-nzhc5\" (UID: \"843e5f1b-1c67-4b12-b617-1b4f293baa0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5"
Nov 04 01:54:59.921132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:54:59.921109    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/843e5f1b-1c67-4b12-b617-1b4f293baa0c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-da8cb04f-nzhc5\" (UID: \"843e5f1b-1c67-4b12-b617-1b4f293baa0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5"
Nov 04 01:55:00.021975 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:00.021947    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/843e5f1b-1c67-4b12-b617-1b4f293baa0c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-da8cb04f-nzhc5\" (UID: \"843e5f1b-1c67-4b12-b617-1b4f293baa0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5"
Nov 04 01:55:00.022143 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:00.022006    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj6ls\" (UniqueName: \"kubernetes.io/projected/843e5f1b-1c67-4b12-b617-1b4f293baa0c-kube-api-access-qj6ls\") pod \"maas-api-revoke-sub-da8cb04f-nzhc5\" (UID: \"843e5f1b-1c67-4b12-b617-1b4f293baa0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5"
Nov 04 01:55:00.022192 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:00.022144    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/843e5f1b-1c67-4b12-b617-1b4f293baa0c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-da8cb04f-nzhc5\" (UID: \"843e5f1b-1c67-4b12-b617-1b4f293baa0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5"
Nov 04 01:55:00.022646 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:00.022625    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/843e5f1b-1c67-4b12-b617-1b4f293baa0c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-da8cb04f-nzhc5\" (UID: \"843e5f1b-1c67-4b12-b617-1b4f293baa0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5"
Nov 04 01:55:00.024503 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:00.024484    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/843e5f1b-1c67-4b12-b617-1b4f293baa0c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-da8cb04f-nzhc5\" (UID: \"843e5f1b-1c67-4b12-b617-1b4f293baa0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5"
Nov 04 01:55:00.083337 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:00.083272    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj6ls\" (UniqueName: \"kubernetes.io/projected/843e5f1b-1c67-4b12-b617-1b4f293baa0c-kube-api-access-qj6ls\") pod \"maas-api-revoke-sub-da8cb04f-nzhc5\" (UID: \"843e5f1b-1c67-4b12-b617-1b4f293baa0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5"
Nov 04 01:55:00.133827 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:00.133795    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5"
Nov 04 01:55:00.170000 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:00.169968    6116 generic.go:356] "Generic (PLEG): container finished" podID="cf7d56e8-d90e-4557-b72d-a3ac56015867" containerID="eeffd11c7335008162a58f11eb7e21122943d788d5216bd753797b0cf4e5a429" exitCode=0
Nov 04 01:55:00.170165 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:00.170058    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr" event={"ID":"cf7d56e8-d90e-4557-b72d-a3ac56015867","Type":"ContainerDied","Data":"eeffd11c7335008162a58f11eb7e21122943d788d5216bd753797b0cf4e5a429"}
Nov 04 01:55:00.170165 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:00.170124    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr" event={"ID":"cf7d56e8-d90e-4557-b72d-a3ac56015867","Type":"ContainerStarted","Data":"d91313cffa01ae3e9d784060c165689f89a16c7b26a8a8319b20f8c9297d5922"}
Nov 04 01:55:00.508680 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:00.508652    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5"]
Nov 04 01:55:00.509386 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:55:00.509367    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod843e5f1b_1c67_4b12_b617_1b4f293baa0c.slice/crio-ae14dc086d051af300b24d77f45fa1effd145e95501f9a2191177573cab5c85c WatchSource:0}: Error finding container ae14dc086d051af300b24d77f45fa1effd145e95501f9a2191177573cab5c85c: Status 404 returned error can't find the container with id ae14dc086d051af300b24d77f45fa1effd145e95501f9a2191177573cab5c85c
Nov 04 01:55:01.178064 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:01.178026    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5" event={"ID":"843e5f1b-1c67-4b12-b617-1b4f293baa0c","Type":"ContainerStarted","Data":"9063bef9f2120c3d8c126f6571483481a1cdbe7edbdf2dad33cb4f04e1a03521"}
Nov 04 01:55:01.178515 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:01.178083    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5" event={"ID":"843e5f1b-1c67-4b12-b617-1b4f293baa0c","Type":"ContainerStarted","Data":"ae14dc086d051af300b24d77f45fa1effd145e95501f9a2191177573cab5c85c"}
Nov 04 01:55:01.294650 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:01.294602    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5" podStartSLOduration=2.294586411 podStartE2EDuration="2.294586411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:54:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:01.293520755 +0000 UTC m=+1186.414798282" watchObservedRunningTime="2026-11-04 01:55:01.294586411 +0000 UTC m=+1186.415863933"
Nov 04 01:55:01.323288 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:01.323265    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr"
Nov 04 01:55:01.450219 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:01.450172    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf7d56e8-d90e-4557-b72d-a3ac56015867-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf7d56e8-d90e-4557-b72d-a3ac56015867-maas-api-xks-ca\") pod \"cf7d56e8-d90e-4557-b72d-a3ac56015867\" (UID: \"cf7d56e8-d90e-4557-b72d-a3ac56015867\") "
Nov 04 01:55:01.450368 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:01.450329    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf7d56e8-d90e-4557-b72d-a3ac56015867-kube-api-access-l5r5r\" (UniqueName: \"kubernetes.io/projected/cf7d56e8-d90e-4557-b72d-a3ac56015867-kube-api-access-l5r5r\") pod \"cf7d56e8-d90e-4557-b72d-a3ac56015867\" (UID: \"cf7d56e8-d90e-4557-b72d-a3ac56015867\") "
Nov 04 01:55:01.450426 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:01.450374    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf7d56e8-d90e-4557-b72d-a3ac56015867-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf7d56e8-d90e-4557-b72d-a3ac56015867-maas-api-openshift-ca\") pod \"cf7d56e8-d90e-4557-b72d-a3ac56015867\" (UID: \"cf7d56e8-d90e-4557-b72d-a3ac56015867\") "
Nov 04 01:55:01.450955 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:01.450926    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf7d56e8-d90e-4557-b72d-a3ac56015867-maas-api-openshift-ca" pod "cf7d56e8-d90e-4557-b72d-a3ac56015867" (UID: "cf7d56e8-d90e-4557-b72d-a3ac56015867"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:55:01.452187 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:01.452151    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf7d56e8-d90e-4557-b72d-a3ac56015867-maas-api-openshift-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:01.455764 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:01.455723    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf7d56e8-d90e-4557-b72d-a3ac56015867-maas-api-xks-ca" pod "cf7d56e8-d90e-4557-b72d-a3ac56015867" (UID: "cf7d56e8-d90e-4557-b72d-a3ac56015867"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:55:01.456060 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:01.456032    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf7d56e8-d90e-4557-b72d-a3ac56015867-kube-api-access-l5r5r" pod "cf7d56e8-d90e-4557-b72d-a3ac56015867" (UID: "cf7d56e8-d90e-4557-b72d-a3ac56015867"). InnerVolumeSpecName "kube-api-access-l5r5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:55:01.553922 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:01.553889    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf7d56e8-d90e-4557-b72d-a3ac56015867-maas-api-xks-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:01.553922 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:01.553917    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5r5r\" (UniqueName: \"kubernetes.io/projected/cf7d56e8-d90e-4557-b72d-a3ac56015867-kube-api-access-l5r5r\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:02.186022 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:02.185986    6116 generic.go:356] "Generic (PLEG): container finished" podID="843e5f1b-1c67-4b12-b617-1b4f293baa0c" containerID="9063bef9f2120c3d8c126f6571483481a1cdbe7edbdf2dad33cb4f04e1a03521" exitCode=0
Nov 04 01:55:02.186435 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:02.186073    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5" event={"ID":"843e5f1b-1c67-4b12-b617-1b4f293baa0c","Type":"ContainerDied","Data":"9063bef9f2120c3d8c126f6571483481a1cdbe7edbdf2dad33cb4f04e1a03521"}
Nov 04 01:55:02.187378 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:02.187356    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr" event={"ID":"cf7d56e8-d90e-4557-b72d-a3ac56015867","Type":"ContainerDied","Data":"d91313cffa01ae3e9d784060c165689f89a16c7b26a8a8319b20f8c9297d5922"}
Nov 04 01:55:02.187378 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:02.187371    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr"
Nov 04 01:55:02.187541 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:02.187391    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d91313cffa01ae3e9d784060c165689f89a16c7b26a8a8319b20f8c9297d5922"
Nov 04 01:55:02.743171 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:02.743137    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"]
Nov 04 01:55:03.347331 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:03.347307    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5"
Nov 04 01:55:03.478634 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:03.478543    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/843e5f1b-1c67-4b12-b617-1b4f293baa0c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/843e5f1b-1c67-4b12-b617-1b4f293baa0c-maas-api-openshift-ca\") pod \"843e5f1b-1c67-4b12-b617-1b4f293baa0c\" (UID: \"843e5f1b-1c67-4b12-b617-1b4f293baa0c\") "
Nov 04 01:55:03.478799 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:03.478650    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/843e5f1b-1c67-4b12-b617-1b4f293baa0c-kube-api-access-qj6ls\" (UniqueName: \"kubernetes.io/projected/843e5f1b-1c67-4b12-b617-1b4f293baa0c-kube-api-access-qj6ls\") pod \"843e5f1b-1c67-4b12-b617-1b4f293baa0c\" (UID: \"843e5f1b-1c67-4b12-b617-1b4f293baa0c\") "
Nov 04 01:55:03.478799 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:03.478674    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/843e5f1b-1c67-4b12-b617-1b4f293baa0c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/843e5f1b-1c67-4b12-b617-1b4f293baa0c-maas-api-xks-ca\") pod \"843e5f1b-1c67-4b12-b617-1b4f293baa0c\" (UID: \"843e5f1b-1c67-4b12-b617-1b4f293baa0c\") "
Nov 04 01:55:03.478964 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:03.478930    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/843e5f1b-1c67-4b12-b617-1b4f293baa0c-maas-api-openshift-ca" pod "843e5f1b-1c67-4b12-b617-1b4f293baa0c" (UID: "843e5f1b-1c67-4b12-b617-1b4f293baa0c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:55:03.479298 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:03.479268    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/843e5f1b-1c67-4b12-b617-1b4f293baa0c-maas-api-openshift-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:03.481431 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:03.481405    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/843e5f1b-1c67-4b12-b617-1b4f293baa0c-maas-api-xks-ca" pod "843e5f1b-1c67-4b12-b617-1b4f293baa0c" (UID: "843e5f1b-1c67-4b12-b617-1b4f293baa0c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:55:03.481431 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:03.481429    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/843e5f1b-1c67-4b12-b617-1b4f293baa0c-kube-api-access-qj6ls" pod "843e5f1b-1c67-4b12-b617-1b4f293baa0c" (UID: "843e5f1b-1c67-4b12-b617-1b4f293baa0c"). InnerVolumeSpecName "kube-api-access-qj6ls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:55:03.580210 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:03.580173    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qj6ls\" (UniqueName: \"kubernetes.io/projected/843e5f1b-1c67-4b12-b617-1b4f293baa0c-kube-api-access-qj6ls\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:03.580210 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:03.580206    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/843e5f1b-1c67-4b12-b617-1b4f293baa0c-maas-api-xks-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:04.203578 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:04.203545    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5"
Nov 04 01:55:04.203578 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:04.203557    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5" event={"ID":"843e5f1b-1c67-4b12-b617-1b4f293baa0c","Type":"ContainerDied","Data":"ae14dc086d051af300b24d77f45fa1effd145e95501f9a2191177573cab5c85c"}
Nov 04 01:55:04.203794 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:04.203608    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae14dc086d051af300b24d77f45fa1effd145e95501f9a2191177573cab5c85c"
Nov 04 01:55:09.411262 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.411227    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd"]
Nov 04 01:55:09.411794 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.411780    6116 cpu_manager.go:414] "containerMap: removing container" podUID="843e5f1b-1c67-4b12-b617-1b4f293baa0c" containerName="revoke-keys"
Nov 04 01:55:09.411837 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.411794    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="843e5f1b-1c67-4b12-b617-1b4f293baa0c" containerName="revoke-keys"
Nov 04 01:55:09.411837 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.411804    6116 cpu_manager.go:414] "containerMap: removing container" podUID="cf7d56e8-d90e-4557-b72d-a3ac56015867" containerName="revoke-keys"
Nov 04 01:55:09.411837 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.411809    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf7d56e8-d90e-4557-b72d-a3ac56015867" containerName="revoke-keys"
Nov 04 01:55:09.411928 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.411917    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf7d56e8-d90e-4557-b72d-a3ac56015867" containerName="revoke-keys"
Nov 04 01:55:09.411928 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.411923    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="843e5f1b-1c67-4b12-b617-1b4f293baa0c" containerName="revoke-keys"
Nov 04 01:55:09.415410 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.415389    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd"
Nov 04 01:55:09.449410 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.449372    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/33bf4adf-572f-4369-9f99-b685fb65f9e0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-166efb82-hrvgd\" (UID: \"33bf4adf-572f-4369-9f99-b685fb65f9e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd"
Nov 04 01:55:09.449627 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.449606    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m56d9\" (UniqueName: \"kubernetes.io/projected/33bf4adf-572f-4369-9f99-b685fb65f9e0-kube-api-access-m56d9\") pod \"maas-api-revoke-sub-166efb82-hrvgd\" (UID: \"33bf4adf-572f-4369-9f99-b685fb65f9e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd"
Nov 04 01:55:09.449740 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.449719    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/33bf4adf-572f-4369-9f99-b685fb65f9e0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-166efb82-hrvgd\" (UID: \"33bf4adf-572f-4369-9f99-b685fb65f9e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd"
Nov 04 01:55:09.468895 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.468868    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:55:09.518634 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.518605    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:55:09.519031 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.519007    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 01:55:09.519156 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.519049    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bj8b6\""
Nov 04 01:55:09.522202 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.522177    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd"]
Nov 04 01:55:09.551255 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.551227    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m56d9\" (UniqueName: \"kubernetes.io/projected/33bf4adf-572f-4369-9f99-b685fb65f9e0-kube-api-access-m56d9\") pod \"maas-api-revoke-sub-166efb82-hrvgd\" (UID: \"33bf4adf-572f-4369-9f99-b685fb65f9e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd"
Nov 04 01:55:09.551392 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.551287    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/33bf4adf-572f-4369-9f99-b685fb65f9e0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-166efb82-hrvgd\" (UID: \"33bf4adf-572f-4369-9f99-b685fb65f9e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd"
Nov 04 01:55:09.551392 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.551365    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/33bf4adf-572f-4369-9f99-b685fb65f9e0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-166efb82-hrvgd\" (UID: \"33bf4adf-572f-4369-9f99-b685fb65f9e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd"
Nov 04 01:55:09.552003 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.551984    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/33bf4adf-572f-4369-9f99-b685fb65f9e0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-166efb82-hrvgd\" (UID: \"33bf4adf-572f-4369-9f99-b685fb65f9e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd"
Nov 04 01:55:09.553715 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.553694    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/33bf4adf-572f-4369-9f99-b685fb65f9e0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-166efb82-hrvgd\" (UID: \"33bf4adf-572f-4369-9f99-b685fb65f9e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd"
Nov 04 01:55:09.611824 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.611791    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m56d9\" (UniqueName: \"kubernetes.io/projected/33bf4adf-572f-4369-9f99-b685fb65f9e0-kube-api-access-m56d9\") pod \"maas-api-revoke-sub-166efb82-hrvgd\" (UID: \"33bf4adf-572f-4369-9f99-b685fb65f9e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd"
Nov 04 01:55:09.736578 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:09.736495    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd"
Nov 04 01:55:10.112837 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:10.112809    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd"]
Nov 04 01:55:10.113097 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:55:10.113069    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33bf4adf_572f_4369_9f99_b685fb65f9e0.slice/crio-5da6f279c56a5bbc159ac4989f5af2c02af4811d9a446a8b988b20705f15c592 WatchSource:0}: Error finding container 5da6f279c56a5bbc159ac4989f5af2c02af4811d9a446a8b988b20705f15c592: Status 404 returned error can't find the container with id 5da6f279c56a5bbc159ac4989f5af2c02af4811d9a446a8b988b20705f15c592
Nov 04 01:55:10.243934 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:10.243900    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd" event={"ID":"33bf4adf-572f-4369-9f99-b685fb65f9e0","Type":"ContainerStarted","Data":"5da6f279c56a5bbc159ac4989f5af2c02af4811d9a446a8b988b20705f15c592"}
Nov 04 01:55:11.251513 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:11.251478    6116 generic.go:356] "Generic (PLEG): container finished" podID="33bf4adf-572f-4369-9f99-b685fb65f9e0" containerID="abc7e831c38620be19f82b6c205ab01f542c7e0391fed7fb01f72411cc95c7cb" exitCode=0
Nov 04 01:55:11.251909 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:11.251611    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd" event={"ID":"33bf4adf-572f-4369-9f99-b685fb65f9e0","Type":"ContainerDied","Data":"abc7e831c38620be19f82b6c205ab01f542c7e0391fed7fb01f72411cc95c7cb"}
Nov 04 01:55:11.992111 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:55:11.992075    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:55:12.413296 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:12.413269    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd"
Nov 04 01:55:12.489686 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:12.489654    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/33bf4adf-572f-4369-9f99-b685fb65f9e0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/33bf4adf-572f-4369-9f99-b685fb65f9e0-maas-api-xks-ca\") pod \"33bf4adf-572f-4369-9f99-b685fb65f9e0\" (UID: \"33bf4adf-572f-4369-9f99-b685fb65f9e0\") "
Nov 04 01:55:12.489890 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:12.489719    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33bf4adf-572f-4369-9f99-b685fb65f9e0-kube-api-access-m56d9\" (UniqueName: \"kubernetes.io/projected/33bf4adf-572f-4369-9f99-b685fb65f9e0-kube-api-access-m56d9\") pod \"33bf4adf-572f-4369-9f99-b685fb65f9e0\" (UID: \"33bf4adf-572f-4369-9f99-b685fb65f9e0\") "
Nov 04 01:55:12.489890 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:12.489829    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33bf4adf-572f-4369-9f99-b685fb65f9e0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/33bf4adf-572f-4369-9f99-b685fb65f9e0-maas-api-openshift-ca\") pod \"33bf4adf-572f-4369-9f99-b685fb65f9e0\" (UID: \"33bf4adf-572f-4369-9f99-b685fb65f9e0\") "
Nov 04 01:55:12.490148 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:12.490116    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33bf4adf-572f-4369-9f99-b685fb65f9e0-maas-api-openshift-ca" pod "33bf4adf-572f-4369-9f99-b685fb65f9e0" (UID: "33bf4adf-572f-4369-9f99-b685fb65f9e0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:55:12.490430 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:12.490412    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/33bf4adf-572f-4369-9f99-b685fb65f9e0-maas-api-openshift-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:12.491996 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:12.491976    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33bf4adf-572f-4369-9f99-b685fb65f9e0-maas-api-xks-ca" pod "33bf4adf-572f-4369-9f99-b685fb65f9e0" (UID: "33bf4adf-572f-4369-9f99-b685fb65f9e0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:55:12.491996 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:12.491981    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33bf4adf-572f-4369-9f99-b685fb65f9e0-kube-api-access-m56d9" pod "33bf4adf-572f-4369-9f99-b685fb65f9e0" (UID: "33bf4adf-572f-4369-9f99-b685fb65f9e0"). InnerVolumeSpecName "kube-api-access-m56d9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:55:12.592236 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:12.592157    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/33bf4adf-572f-4369-9f99-b685fb65f9e0-maas-api-xks-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:12.592236 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:12.592185    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m56d9\" (UniqueName: \"kubernetes.io/projected/33bf4adf-572f-4369-9f99-b685fb65f9e0-kube-api-access-m56d9\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:13.265282 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:13.265245    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd" event={"ID":"33bf4adf-572f-4369-9f99-b685fb65f9e0","Type":"ContainerDied","Data":"5da6f279c56a5bbc159ac4989f5af2c02af4811d9a446a8b988b20705f15c592"}
Nov 04 01:55:13.265492 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:13.265285    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd"
Nov 04 01:55:13.265492 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:13.265305    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5da6f279c56a5bbc159ac4989f5af2c02af4811d9a446a8b988b20705f15c592"
Nov 04 01:55:14.012589 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:55:14.012558    6116 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-84cffd7d1248: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:55:14.012589 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:55:14.012584    6116 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:55:14.013033 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:55:14.012639    6116 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials podName:233cb7a1-2c92-4544-afe5-5aae953288e1 nodeName:}" failed. No retries permitted until 2026-11-04 01:57:16.0126225 +0000 UTC m=+1321.133900021 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials") pod "payload-processing-external-model-5497dccdcc-bbs4k" (UID: "233cb7a1-2c92-4544-afe5-5aae953288e1") : secret "agc-cred-84cffd7d1248" not found
Nov 04 01:55:15.240142 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:15.240115    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/prometheus/0.log"
Nov 04 01:55:15.241059 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:15.241039    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/prometheus/0.log"
Nov 04 01:55:15.251121 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:15.251099    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6lfbv_192e2fec-fefb-4bc5-9516-ed7464963a10/ovn-acl-logging/0.log"
Nov 04 01:55:15.252112 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:15.252089    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6lfbv_192e2fec-fefb-4bc5-9516-ed7464963a10/ovn-acl-logging/0.log"
Nov 04 01:55:15.736028 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:15.735994    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl"]
Nov 04 01:55:17.764263 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:17.764226    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl" podUID="b42d0649-0316-4a47-af9f-e271c5b7068f" containerName="main" containerID="cri-o://933a0891f43c9126aa89364737c99eeac432943022a60f7a6708b38c66abb15d" gracePeriod=45
Nov 04 01:55:17.834617 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:17.834577    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:55:18.040496 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.039424    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:55:18.135478 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.135433    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"]
Nov 04 01:55:18.180711 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.180685    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-tmp-dir\") pod \"b42d0649-0316-4a47-af9f-e271c5b7068f\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") "
Nov 04 01:55:18.180886 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.180732    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b42d0649-0316-4a47-af9f-e271c5b7068f-tls-certs\" (UniqueName: \"kubernetes.io/secret/b42d0649-0316-4a47-af9f-e271c5b7068f-tls-certs\") pod \"b42d0649-0316-4a47-af9f-e271c5b7068f\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") "
Nov 04 01:55:18.180886 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.180813    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-model-cache\") pod \"b42d0649-0316-4a47-af9f-e271c5b7068f\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") "
Nov 04 01:55:18.180886 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.180867    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-dshm\") pod \"b42d0649-0316-4a47-af9f-e271c5b7068f\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") "
Nov 04 01:55:18.181063 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.180925    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b42d0649-0316-4a47-af9f-e271c5b7068f-kube-api-access-bfvp5\" (UniqueName: \"kubernetes.io/projected/b42d0649-0316-4a47-af9f-e271c5b7068f-kube-api-access-bfvp5\") pod \"b42d0649-0316-4a47-af9f-e271c5b7068f\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") "
Nov 04 01:55:18.181063 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.180939    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-tmp-dir" pod "b42d0649-0316-4a47-af9f-e271c5b7068f" (UID: "b42d0649-0316-4a47-af9f-e271c5b7068f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:55:18.181063 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.180970    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-home\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-home\") pod \"b42d0649-0316-4a47-af9f-e271c5b7068f\" (UID: \"b42d0649-0316-4a47-af9f-e271c5b7068f\") "
Nov 04 01:55:18.181207 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.181064    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-model-cache" pod "b42d0649-0316-4a47-af9f-e271c5b7068f" (UID: "b42d0649-0316-4a47-af9f-e271c5b7068f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:55:18.181281 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.181258    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-home" pod "b42d0649-0316-4a47-af9f-e271c5b7068f" (UID: "b42d0649-0316-4a47-af9f-e271c5b7068f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:55:18.181527 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.181506    6116 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-home\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:18.181635 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.181529    6116 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-tmp-dir\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:18.181635 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.181541    6116 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-model-cache\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:18.183604 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.183565    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b42d0649-0316-4a47-af9f-e271c5b7068f-tls-certs" pod "b42d0649-0316-4a47-af9f-e271c5b7068f" (UID: "b42d0649-0316-4a47-af9f-e271c5b7068f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:55:18.183795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.183664    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b42d0649-0316-4a47-af9f-e271c5b7068f-kube-api-access-bfvp5" pod "b42d0649-0316-4a47-af9f-e271c5b7068f" (UID: "b42d0649-0316-4a47-af9f-e271c5b7068f"). InnerVolumeSpecName "kube-api-access-bfvp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:55:18.185014 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.184993    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-dshm" pod "b42d0649-0316-4a47-af9f-e271c5b7068f" (UID: "b42d0649-0316-4a47-af9f-e271c5b7068f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:55:18.283104 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.283008    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfvp5\" (UniqueName: \"kubernetes.io/projected/b42d0649-0316-4a47-af9f-e271c5b7068f-kube-api-access-bfvp5\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:18.283104 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.283053    6116 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b42d0649-0316-4a47-af9f-e271c5b7068f-tls-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:18.283104 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.283066    6116 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b42d0649-0316-4a47-af9f-e271c5b7068f-dshm\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:18.300047 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.300018    6116 generic.go:356] "Generic (PLEG): container finished" podID="b42d0649-0316-4a47-af9f-e271c5b7068f" containerID="933a0891f43c9126aa89364737c99eeac432943022a60f7a6708b38c66abb15d" exitCode=0
Nov 04 01:55:18.300211 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.300060    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"
Nov 04 01:55:18.300211 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.300103    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl" event={"ID":"b42d0649-0316-4a47-af9f-e271c5b7068f","Type":"ContainerDied","Data":"933a0891f43c9126aa89364737c99eeac432943022a60f7a6708b38c66abb15d"}
Nov 04 01:55:18.300211 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.300146    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl" event={"ID":"b42d0649-0316-4a47-af9f-e271c5b7068f","Type":"ContainerDied","Data":"69d874f1595d82f7c8ddd0b60a657d7d58cb6c378a0270cf453bd17707e4454e"}
Nov 04 01:55:18.300211 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.300166    6116 scope.go:122] "RemoveContainer" containerID="933a0891f43c9126aa89364737c99eeac432943022a60f7a6708b38c66abb15d"
Nov 04 01:55:18.310590 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.310572    6116 scope.go:122] "RemoveContainer" containerID="933a0891f43c9126aa89364737c99eeac432943022a60f7a6708b38c66abb15d"
Nov 04 01:55:18.310880 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:55:18.310858    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"933a0891f43c9126aa89364737c99eeac432943022a60f7a6708b38c66abb15d\": container with ID starting with 933a0891f43c9126aa89364737c99eeac432943022a60f7a6708b38c66abb15d not found: ID does not exist" containerID="933a0891f43c9126aa89364737c99eeac432943022a60f7a6708b38c66abb15d"
Nov 04 01:55:18.310916 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.310889    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"933a0891f43c9126aa89364737c99eeac432943022a60f7a6708b38c66abb15d"} err="failed to get container status \"933a0891f43c9126aa89364737c99eeac432943022a60f7a6708b38c66abb15d\": rpc error: code = NotFound desc = could not find container \"933a0891f43c9126aa89364737c99eeac432943022a60f7a6708b38c66abb15d\": container with ID starting with 933a0891f43c9126aa89364737c99eeac432943022a60f7a6708b38c66abb15d not found: ID does not exist"
Nov 04 01:55:18.480250 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.480213    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"]
Nov 04 01:55:18.486563 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.486534    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl"]
Nov 04 01:55:18.972025 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:18.971976    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-795045/premium-w1-795045-kserve-86d9bc5fc5-ffhvl" podUID="b42d0649-0316-4a47-af9f-e271c5b7068f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 04 01:55:19.028631 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:19.028600    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b42d0649-0316-4a47-af9f-e271c5b7068f" path="/var/lib/kubelet/pods/b42d0649-0316-4a47-af9f-e271c5b7068f/volumes"
Nov 04 01:55:20.736418 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:20.736374    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl" podUID="02295850-87a3-45a0-aa2e-1f42985f6c79" containerName="payload-pre-processing" containerID="cri-o://9cb6c726bc5a361540d13d9ee6e1c13b1ef3a0c0d45bd692b16d083b8d0eb85b" gracePeriod=25
Nov 04 01:55:23.136640 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:23.136582    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn" podUID="fee5848c-8694-4d7e-96d9-7536e13022a3" containerName="payload-processing" containerID="cri-o://277911ca153e731f420a6c8041e9cfe69913be1aa22f03d7cc9867eec5eb61a2" gracePeriod=25
Nov 04 01:55:23.335336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:23.335307    6116 generic.go:356] "Generic (PLEG): container finished" podID="fee5848c-8694-4d7e-96d9-7536e13022a3" containerID="277911ca153e731f420a6c8041e9cfe69913be1aa22f03d7cc9867eec5eb61a2" exitCode=0
Nov 04 01:55:23.335503 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:23.335377    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn" event={"ID":"fee5848c-8694-4d7e-96d9-7536e13022a3","Type":"ContainerDied","Data":"277911ca153e731f420a6c8041e9cfe69913be1aa22f03d7cc9867eec5eb61a2"}
Nov 04 01:55:23.416337 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:23.416313    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"
Nov 04 01:55:23.563264 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:23.563229    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fee5848c-8694-4d7e-96d9-7536e13022a3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fee5848c-8694-4d7e-96d9-7536e13022a3-plugins-config-volume\") pod \"fee5848c-8694-4d7e-96d9-7536e13022a3\" (UID: \"fee5848c-8694-4d7e-96d9-7536e13022a3\") "
Nov 04 01:55:23.563489 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:23.563284    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fee5848c-8694-4d7e-96d9-7536e13022a3-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/fee5848c-8694-4d7e-96d9-7536e13022a3-routing-overlay-volume\") pod \"fee5848c-8694-4d7e-96d9-7536e13022a3\" (UID: \"fee5848c-8694-4d7e-96d9-7536e13022a3\") "
Nov 04 01:55:23.563489 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:23.563332    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fee5848c-8694-4d7e-96d9-7536e13022a3-provider-credentials\" (UniqueName: \"kubernetes.io/projected/fee5848c-8694-4d7e-96d9-7536e13022a3-provider-credentials\") pod \"fee5848c-8694-4d7e-96d9-7536e13022a3\" (UID: \"fee5848c-8694-4d7e-96d9-7536e13022a3\") "
Nov 04 01:55:23.563489 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:23.563417    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fee5848c-8694-4d7e-96d9-7536e13022a3-kube-api-access-tblrn\" (UniqueName: \"kubernetes.io/projected/fee5848c-8694-4d7e-96d9-7536e13022a3-kube-api-access-tblrn\") pod \"fee5848c-8694-4d7e-96d9-7536e13022a3\" (UID: \"fee5848c-8694-4d7e-96d9-7536e13022a3\") "
Nov 04 01:55:23.563761 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:23.563733    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fee5848c-8694-4d7e-96d9-7536e13022a3-plugins-config-volume" pod "fee5848c-8694-4d7e-96d9-7536e13022a3" (UID: "fee5848c-8694-4d7e-96d9-7536e13022a3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:55:23.563925 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:23.563743    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fee5848c-8694-4d7e-96d9-7536e13022a3-routing-overlay-volume" pod "fee5848c-8694-4d7e-96d9-7536e13022a3" (UID: "fee5848c-8694-4d7e-96d9-7536e13022a3"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:55:23.564179 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:23.564161    6116 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fee5848c-8694-4d7e-96d9-7536e13022a3-plugins-config-volume\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:23.564179 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:23.564180    6116 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/fee5848c-8694-4d7e-96d9-7536e13022a3-routing-overlay-volume\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:23.565922 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:23.565898    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fee5848c-8694-4d7e-96d9-7536e13022a3-kube-api-access-tblrn" pod "fee5848c-8694-4d7e-96d9-7536e13022a3" (UID: "fee5848c-8694-4d7e-96d9-7536e13022a3"). InnerVolumeSpecName "kube-api-access-tblrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:55:23.566069 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:23.566052    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fee5848c-8694-4d7e-96d9-7536e13022a3-provider-credentials" pod "fee5848c-8694-4d7e-96d9-7536e13022a3" (UID: "fee5848c-8694-4d7e-96d9-7536e13022a3"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:55:23.665022 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:23.664991    6116 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/fee5848c-8694-4d7e-96d9-7536e13022a3-provider-credentials\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:23.665022 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:23.665016    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tblrn\" (UniqueName: \"kubernetes.io/projected/fee5848c-8694-4d7e-96d9-7536e13022a3-kube-api-access-tblrn\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:24.343640 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:24.343600    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn" event={"ID":"fee5848c-8694-4d7e-96d9-7536e13022a3","Type":"ContainerDied","Data":"c8c1158472f36bef7dc8cba642c4ccba5da0aba203ca38731f85ab553146c736"}
Nov 04 01:55:24.343640 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:24.343634    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"
Nov 04 01:55:24.344115 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:24.343652    6116 scope.go:122] "RemoveContainer" containerID="277911ca153e731f420a6c8041e9cfe69913be1aa22f03d7cc9867eec5eb61a2"
Nov 04 01:55:24.508698 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:24.508669    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"]
Nov 04 01:55:24.558848 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:24.558817    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-c0ab5f/payload-processing-external-model-e2e-extmod-b-c0ab5f-6fdbhljgn"]
Nov 04 01:55:25.028648 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:25.028614    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fee5848c-8694-4d7e-96d9-7536e13022a3" path="/var/lib/kubelet/pods/fee5848c-8694-4d7e-96d9-7536e13022a3/volumes"
Nov 04 01:55:26.069056 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:26.069020    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl" podUID="02295850-87a3-45a0-aa2e-1f42985f6c79" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:55:36.069328 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:36.069291    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl" podUID="02295850-87a3-45a0-aa2e-1f42985f6c79" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:55:41.041406 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:41.041380    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl"
Nov 04 01:55:41.199620 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:41.199581    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/02295850-87a3-45a0-aa2e-1f42985f6c79-kube-api-access-t6dh5\" (UniqueName: \"kubernetes.io/projected/02295850-87a3-45a0-aa2e-1f42985f6c79-kube-api-access-t6dh5\") pod \"02295850-87a3-45a0-aa2e-1f42985f6c79\" (UID: \"02295850-87a3-45a0-aa2e-1f42985f6c79\") "
Nov 04 01:55:41.199880 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:41.199863    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/02295850-87a3-45a0-aa2e-1f42985f6c79-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02295850-87a3-45a0-aa2e-1f42985f6c79-plugins-config-volume\") pod \"02295850-87a3-45a0-aa2e-1f42985f6c79\" (UID: \"02295850-87a3-45a0-aa2e-1f42985f6c79\") "
Nov 04 01:55:41.200163 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:41.200140    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02295850-87a3-45a0-aa2e-1f42985f6c79-plugins-config-volume" pod "02295850-87a3-45a0-aa2e-1f42985f6c79" (UID: "02295850-87a3-45a0-aa2e-1f42985f6c79"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:55:41.200315 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:41.200302    6116 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02295850-87a3-45a0-aa2e-1f42985f6c79-plugins-config-volume\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:41.202345 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:41.202317    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02295850-87a3-45a0-aa2e-1f42985f6c79-kube-api-access-t6dh5" pod "02295850-87a3-45a0-aa2e-1f42985f6c79" (UID: "02295850-87a3-45a0-aa2e-1f42985f6c79"). InnerVolumeSpecName "kube-api-access-t6dh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:55:41.302082 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:41.302000    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6dh5\" (UniqueName: \"kubernetes.io/projected/02295850-87a3-45a0-aa2e-1f42985f6c79-kube-api-access-t6dh5\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:55:41.488747 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:41.488718    6116 generic.go:356] "Generic (PLEG): container finished" podID="02295850-87a3-45a0-aa2e-1f42985f6c79" containerID="9cb6c726bc5a361540d13d9ee6e1c13b1ef3a0c0d45bd692b16d083b8d0eb85b" exitCode=0
Nov 04 01:55:41.488933 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:41.488763    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl"
Nov 04 01:55:41.488933 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:41.488809    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl" event={"ID":"02295850-87a3-45a0-aa2e-1f42985f6c79","Type":"ContainerDied","Data":"9cb6c726bc5a361540d13d9ee6e1c13b1ef3a0c0d45bd692b16d083b8d0eb85b"}
Nov 04 01:55:41.488933 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:41.488861    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl" event={"ID":"02295850-87a3-45a0-aa2e-1f42985f6c79","Type":"ContainerDied","Data":"31c43cc38d960626643f402e6d01fd404483ec482b5011f0a5bb2e83447fb5af"}
Nov 04 01:55:41.488933 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:41.488887    6116 scope.go:122] "RemoveContainer" containerID="9cb6c726bc5a361540d13d9ee6e1c13b1ef3a0c0d45bd692b16d083b8d0eb85b"
Nov 04 01:55:41.498886 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:41.498870    6116 scope.go:122] "RemoveContainer" containerID="9cb6c726bc5a361540d13d9ee6e1c13b1ef3a0c0d45bd692b16d083b8d0eb85b"
Nov 04 01:55:41.499169 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:55:41.499150    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cb6c726bc5a361540d13d9ee6e1c13b1ef3a0c0d45bd692b16d083b8d0eb85b\": container with ID starting with 9cb6c726bc5a361540d13d9ee6e1c13b1ef3a0c0d45bd692b16d083b8d0eb85b not found: ID does not exist" containerID="9cb6c726bc5a361540d13d9ee6e1c13b1ef3a0c0d45bd692b16d083b8d0eb85b"
Nov 04 01:55:41.499218 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:41.499176    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cb6c726bc5a361540d13d9ee6e1c13b1ef3a0c0d45bd692b16d083b8d0eb85b"} err="failed to get container status \"9cb6c726bc5a361540d13d9ee6e1c13b1ef3a0c0d45bd692b16d083b8d0eb85b\": rpc error: code = NotFound desc = could not find container \"9cb6c726bc5a361540d13d9ee6e1c13b1ef3a0c0d45bd692b16d083b8d0eb85b\": container with ID starting with 9cb6c726bc5a361540d13d9ee6e1c13b1ef3a0c0d45bd692b16d083b8d0eb85b not found: ID does not exist"
Nov 04 01:55:41.672841 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:41.672792    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl"]
Nov 04 01:55:41.675788 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:41.675763    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl"]
Nov 04 01:55:41.998383 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:55:41.998302    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:55:43.027528 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:43.027481    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="02295850-87a3-45a0-aa2e-1f42985f6c79" path="/var/lib/kubelet/pods/02295850-87a3-45a0-aa2e-1f42985f6c79/volumes"
Nov 04 01:55:46.941636 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:46.941592    6116 patch_prober.go:29] interesting pod/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:55:46.942061 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:46.941693    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6" podUID="a649747c-ccad-4629-8be4-43fc827eb3f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:55:49.549508 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:49.549474    6116 generic.go:356] "Generic (PLEG): container finished" podID="a649747c-ccad-4629-8be4-43fc827eb3f9" containerID="589c975d481f8234f9faf58c83e27b0a5a1f8f3d2dc87189ba1c69ea1cfd5f2a" exitCode=0
Nov 04 01:55:49.550062 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:49.549542    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6" event={"ID":"a649747c-ccad-4629-8be4-43fc827eb3f9","Type":"ContainerDied","Data":"589c975d481f8234f9faf58c83e27b0a5a1f8f3d2dc87189ba1c69ea1cfd5f2a"}
Nov 04 01:55:49.550112 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:49.550063    6116 scope.go:122] "RemoveContainer" containerID="589c975d481f8234f9faf58c83e27b0a5a1f8f3d2dc87189ba1c69ea1cfd5f2a"
Nov 04 01:55:49.551884 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:49.551847    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:55:49.551986 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:49.551948    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:55:49.552045 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:49.552002    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:55:50.557554 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:50.557522    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6" event={"ID":"a649747c-ccad-4629-8be4-43fc827eb3f9","Type":"ContainerStarted","Data":"7810a6f5c8c7c6c57582c3cc5da64104795d7970d7444b97fcee6c686d30c92c"}
Nov 04 01:55:51.370539 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:51.369114    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:55:51.370539 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:51.369318    6116 patch_prober.go:29] interesting pod/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:55:51.370539 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:51.369364    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6" podUID="a649747c-ccad-4629-8be4-43fc827eb3f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Nov 04 01:55:52.370075 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:52.370038    6116 patch_prober.go:29] interesting pod/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:55:52.370709 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:52.370670    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6" podUID="a649747c-ccad-4629-8be4-43fc827eb3f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Nov 04 01:55:52.620170 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:52.620094    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:55:53.368987 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:53.368944    6116 patch_prober.go:29] interesting pod/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:55:53.369417 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:53.369386    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6" podUID="a649747c-ccad-4629-8be4-43fc827eb3f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Nov 04 01:55:54.369475 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:54.368312    6116 patch_prober.go:29] interesting pod/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:55:54.369475 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:54.368382    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6" podUID="a649747c-ccad-4629-8be4-43fc827eb3f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Nov 04 01:55:55.370837 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:55.370797    6116 patch_prober.go:29] interesting pod/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Nov 04 01:55:55.371634 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:55.371602    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6" podUID="a649747c-ccad-4629-8be4-43fc827eb3f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Nov 04 01:55:56.578096 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:56.578061    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:55:56.578594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:56.578409    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:55:56.579598 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:56.579575    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:55:57.788668 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:57.788631    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"]
Nov 04 01:55:57.789412 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:57.788859    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4" podUID="5e55d400-e27f-45fd-b79c-50f9ca6c7da3" containerName="istio-proxy" containerID="cri-o://f77f96fd3f72210521c3ea594e3ee6060df08975b375279ca59065d18af02886" gracePeriod=30
Nov 04 01:55:58.829119 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:58.829082    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"]
Nov 04 01:55:58.829575 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:55:58.829309    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6" podUID="35deac99-14d0-416d-9c8d-c8abb03f5f7d" containerName="istio-proxy" containerID="cri-o://6421160f46b28a73eef60df1aa736b855a28f0bc93c3199f948cc69ba3514c36" gracePeriod=30
Nov 04 01:56:02.328406 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:02.328367    6116 patch_prober.go:29] interesting pod/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:56:02.328835 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:02.328479    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4" podUID="5e55d400-e27f-45fd-b79c-50f9ca6c7da3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:56:03.077137 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.077078    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:56:03.220494 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.220438    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-token\" (UniqueName: \"kubernetes.io/projected/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-token\") pod \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") "
Nov 04 01:56:03.220661 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.220514    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istiod-ca-cert\") pod \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") "
Nov 04 01:56:03.220661 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.220552    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-workload-certs\") pod \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") "
Nov 04 01:56:03.220661 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.220642    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-data\") pod \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") "
Nov 04 01:56:03.220772 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.220674    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-kube-api-access-44zkb\" (UniqueName: \"kubernetes.io/projected/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-kube-api-access-44zkb\") pod \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") "
Nov 04 01:56:03.220772 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.220743    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-envoy\") pod \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") "
Nov 04 01:56:03.220877 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.220816    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-podinfo\") pod \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") "
Nov 04 01:56:03.220931 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.220879    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-credential-socket\") pod \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") "
Nov 04 01:56:03.220931 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.220909    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-workload-certs" pod "5e55d400-e27f-45fd-b79c-50f9ca6c7da3" (UID: "5e55d400-e27f-45fd-b79c-50f9ca6c7da3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:03.221037 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.220928    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-data" pod "5e55d400-e27f-45fd-b79c-50f9ca6c7da3" (UID: "5e55d400-e27f-45fd-b79c-50f9ca6c7da3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:03.221037 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.220960    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-workload-socket\") pod \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\" (UID: \"5e55d400-e27f-45fd-b79c-50f9ca6c7da3\") "
Nov 04 01:56:03.221037 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.220979    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istiod-ca-cert" pod "5e55d400-e27f-45fd-b79c-50f9ca6c7da3" (UID: "5e55d400-e27f-45fd-b79c-50f9ca6c7da3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:03.221285 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.221212    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-workload-socket" pod "5e55d400-e27f-45fd-b79c-50f9ca6c7da3" (UID: "5e55d400-e27f-45fd-b79c-50f9ca6c7da3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:03.221772 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.221746    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-credential-socket" pod "5e55d400-e27f-45fd-b79c-50f9ca6c7da3" (UID: "5e55d400-e27f-45fd-b79c-50f9ca6c7da3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:03.221772 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.221767    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:03.221924 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.221789    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:03.221924 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.221804    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:03.221924 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.221820    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:03.222935 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.222911    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-token" pod "5e55d400-e27f-45fd-b79c-50f9ca6c7da3" (UID: "5e55d400-e27f-45fd-b79c-50f9ca6c7da3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:03.223050 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.222970    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-podinfo" pod "5e55d400-e27f-45fd-b79c-50f9ca6c7da3" (UID: "5e55d400-e27f-45fd-b79c-50f9ca6c7da3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:56:03.223095 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.223084    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-kube-api-access-44zkb" pod "5e55d400-e27f-45fd-b79c-50f9ca6c7da3" (UID: "5e55d400-e27f-45fd-b79c-50f9ca6c7da3"). InnerVolumeSpecName "kube-api-access-44zkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:03.223132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.223095    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-envoy" pod "5e55d400-e27f-45fd-b79c-50f9ca6c7da3" (UID: "5e55d400-e27f-45fd-b79c-50f9ca6c7da3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:03.323393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.323362    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:03.323393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.323391    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:03.323628 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.323405    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:03.323628 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.323417    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:03.323628 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.323431    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44zkb\" (UniqueName: \"kubernetes.io/projected/5e55d400-e27f-45fd-b79c-50f9ca6c7da3-kube-api-access-44zkb\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:03.658499 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.658435    6116 generic.go:356] "Generic (PLEG): container finished" podID="5e55d400-e27f-45fd-b79c-50f9ca6c7da3" containerID="f77f96fd3f72210521c3ea594e3ee6060df08975b375279ca59065d18af02886" exitCode=0
Nov 04 01:56:03.658991 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.658530    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"
Nov 04 01:56:03.658991 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.658526    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4" event={"ID":"5e55d400-e27f-45fd-b79c-50f9ca6c7da3","Type":"ContainerDied","Data":"f77f96fd3f72210521c3ea594e3ee6060df08975b375279ca59065d18af02886"}
Nov 04 01:56:03.658991 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.658589    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4" event={"ID":"5e55d400-e27f-45fd-b79c-50f9ca6c7da3","Type":"ContainerDied","Data":"5407e7f33f35e088352a20f038a609280a5aa081a4cb081596a0741baba473dc"}
Nov 04 01:56:03.658991 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.658610    6116 scope.go:122] "RemoveContainer" containerID="f77f96fd3f72210521c3ea594e3ee6060df08975b375279ca59065d18af02886"
Nov 04 01:56:03.669884 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.669861    6116 scope.go:122] "RemoveContainer" containerID="f77f96fd3f72210521c3ea594e3ee6060df08975b375279ca59065d18af02886"
Nov 04 01:56:03.670204 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:56:03.670182    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f77f96fd3f72210521c3ea594e3ee6060df08975b375279ca59065d18af02886\": container with ID starting with f77f96fd3f72210521c3ea594e3ee6060df08975b375279ca59065d18af02886 not found: ID does not exist" containerID="f77f96fd3f72210521c3ea594e3ee6060df08975b375279ca59065d18af02886"
Nov 04 01:56:03.670249 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.670213    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f77f96fd3f72210521c3ea594e3ee6060df08975b375279ca59065d18af02886"} err="failed to get container status \"f77f96fd3f72210521c3ea594e3ee6060df08975b375279ca59065d18af02886\": rpc error: code = NotFound desc = could not find container \"f77f96fd3f72210521c3ea594e3ee6060df08975b375279ca59065d18af02886\": container with ID starting with f77f96fd3f72210521c3ea594e3ee6060df08975b375279ca59065d18af02886 not found: ID does not exist"
Nov 04 01:56:03.824016 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.823975    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"]
Nov 04 01:56:03.873489 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:03.873423    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4"]
Nov 04 01:56:04.099360 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.099337    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:56:04.234916 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.234822    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-token\" (UniqueName: \"kubernetes.io/projected/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-token\") pod \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") "
Nov 04 01:56:04.234916 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.234904    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35deac99-14d0-416d-9c8d-c8abb03f5f7d-kube-api-access-tqvhs\" (UniqueName: \"kubernetes.io/projected/35deac99-14d0-416d-9c8d-c8abb03f5f7d-kube-api-access-tqvhs\") pod \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") "
Nov 04 01:56:04.235135 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.234937    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-data\") pod \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") "
Nov 04 01:56:04.235135 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.234977    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-credential-socket\") pod \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") "
Nov 04 01:56:04.235135 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.235059    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-workload-socket\") pod \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") "
Nov 04 01:56:04.235296 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.235136    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-podinfo\") pod \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") "
Nov 04 01:56:04.235296 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.235173    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-envoy\") pod \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") "
Nov 04 01:56:04.235296 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.235207    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istiod-ca-cert\") pod \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") "
Nov 04 01:56:04.235296 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.235269    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-workload-certs\") pod \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\" (UID: \"35deac99-14d0-416d-9c8d-c8abb03f5f7d\") "
Nov 04 01:56:04.235517 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.235284    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-data" pod "35deac99-14d0-416d-9c8d-c8abb03f5f7d" (UID: "35deac99-14d0-416d-9c8d-c8abb03f5f7d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:04.235517 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.235433    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-workload-socket" pod "35deac99-14d0-416d-9c8d-c8abb03f5f7d" (UID: "35deac99-14d0-416d-9c8d-c8abb03f5f7d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:04.235934 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.235630    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-credential-socket" pod "35deac99-14d0-416d-9c8d-c8abb03f5f7d" (UID: "35deac99-14d0-416d-9c8d-c8abb03f5f7d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:04.235934 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.235812    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-workload-certs" pod "35deac99-14d0-416d-9c8d-c8abb03f5f7d" (UID: "35deac99-14d0-416d-9c8d-c8abb03f5f7d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:04.235934 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.235898    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:04.235934 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.235914    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:04.235934 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.235937    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:04.236237 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.235949    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:04.236237 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.235910    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istiod-ca-cert" pod "35deac99-14d0-416d-9c8d-c8abb03f5f7d" (UID: "35deac99-14d0-416d-9c8d-c8abb03f5f7d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:04.240200 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.240172    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-token" pod "35deac99-14d0-416d-9c8d-c8abb03f5f7d" (UID: "35deac99-14d0-416d-9c8d-c8abb03f5f7d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:04.247629 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.247589    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-envoy" pod "35deac99-14d0-416d-9c8d-c8abb03f5f7d" (UID: "35deac99-14d0-416d-9c8d-c8abb03f5f7d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:04.247629 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.247605    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35deac99-14d0-416d-9c8d-c8abb03f5f7d-kube-api-access-tqvhs" pod "35deac99-14d0-416d-9c8d-c8abb03f5f7d" (UID: "35deac99-14d0-416d-9c8d-c8abb03f5f7d"). InnerVolumeSpecName "kube-api-access-tqvhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:04.248916 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.248841    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-podinfo" pod "35deac99-14d0-416d-9c8d-c8abb03f5f7d" (UID: "35deac99-14d0-416d-9c8d-c8abb03f5f7d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:56:04.336705 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.336669    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:04.336705 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.336698    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqvhs\" (UniqueName: \"kubernetes.io/projected/35deac99-14d0-416d-9c8d-c8abb03f5f7d-kube-api-access-tqvhs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:04.336905 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.336713    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:04.336905 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.336726    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:04.336905 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.336735    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35deac99-14d0-416d-9c8d-c8abb03f5f7d-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:04.667384 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.667350    6116 generic.go:356] "Generic (PLEG): container finished" podID="35deac99-14d0-416d-9c8d-c8abb03f5f7d" containerID="6421160f46b28a73eef60df1aa736b855a28f0bc93c3199f948cc69ba3514c36" exitCode=0
Nov 04 01:56:04.667927 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.667412    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"
Nov 04 01:56:04.667927 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.667423    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6" event={"ID":"35deac99-14d0-416d-9c8d-c8abb03f5f7d","Type":"ContainerDied","Data":"6421160f46b28a73eef60df1aa736b855a28f0bc93c3199f948cc69ba3514c36"}
Nov 04 01:56:04.667927 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.667500    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6" event={"ID":"35deac99-14d0-416d-9c8d-c8abb03f5f7d","Type":"ContainerDied","Data":"e9a962d572b0f1a463173acb92dd3b236809b310633ff2992dc9002290a7a19a"}
Nov 04 01:56:04.667927 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.667521    6116 scope.go:122] "RemoveContainer" containerID="6421160f46b28a73eef60df1aa736b855a28f0bc93c3199f948cc69ba3514c36"
Nov 04 01:56:04.677599 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.677576    6116 scope.go:122] "RemoveContainer" containerID="ef2cd128dacca8851fb28b42841f1f650dfc53652a7704966971dae8a36b09de"
Nov 04 01:56:04.686649 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.686626    6116 scope.go:122] "RemoveContainer" containerID="6421160f46b28a73eef60df1aa736b855a28f0bc93c3199f948cc69ba3514c36"
Nov 04 01:56:04.686890 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:56:04.686871    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6421160f46b28a73eef60df1aa736b855a28f0bc93c3199f948cc69ba3514c36\": container with ID starting with 6421160f46b28a73eef60df1aa736b855a28f0bc93c3199f948cc69ba3514c36 not found: ID does not exist" containerID="6421160f46b28a73eef60df1aa736b855a28f0bc93c3199f948cc69ba3514c36"
Nov 04 01:56:04.686939 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.686900    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6421160f46b28a73eef60df1aa736b855a28f0bc93c3199f948cc69ba3514c36"} err="failed to get container status \"6421160f46b28a73eef60df1aa736b855a28f0bc93c3199f948cc69ba3514c36\": rpc error: code = NotFound desc = could not find container \"6421160f46b28a73eef60df1aa736b855a28f0bc93c3199f948cc69ba3514c36\": container with ID starting with 6421160f46b28a73eef60df1aa736b855a28f0bc93c3199f948cc69ba3514c36 not found: ID does not exist"
Nov 04 01:56:04.686939 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.686917    6116 scope.go:122] "RemoveContainer" containerID="ef2cd128dacca8851fb28b42841f1f650dfc53652a7704966971dae8a36b09de"
Nov 04 01:56:04.687134 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:56:04.687118    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef2cd128dacca8851fb28b42841f1f650dfc53652a7704966971dae8a36b09de\": container with ID starting with ef2cd128dacca8851fb28b42841f1f650dfc53652a7704966971dae8a36b09de not found: ID does not exist" containerID="ef2cd128dacca8851fb28b42841f1f650dfc53652a7704966971dae8a36b09de"
Nov 04 01:56:04.687180 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.687137    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef2cd128dacca8851fb28b42841f1f650dfc53652a7704966971dae8a36b09de"} err="failed to get container status \"ef2cd128dacca8851fb28b42841f1f650dfc53652a7704966971dae8a36b09de\": rpc error: code = NotFound desc = could not find container \"ef2cd128dacca8851fb28b42841f1f650dfc53652a7704966971dae8a36b09de\": container with ID starting with ef2cd128dacca8851fb28b42841f1f650dfc53652a7704966971dae8a36b09de not found: ID does not exist"
Nov 04 01:56:04.832265 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.832236    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"]
Nov 04 01:56:04.881896 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:04.881862    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6"]
Nov 04 01:56:05.027999 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:05.027923    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="35deac99-14d0-416d-9c8d-c8abb03f5f7d" path="/var/lib/kubelet/pods/35deac99-14d0-416d-9c8d-c8abb03f5f7d/volumes"
Nov 04 01:56:05.028393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:05.028370    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e55d400-e27f-45fd-b79c-50f9ca6c7da3" path="/var/lib/kubelet/pods/5e55d400-e27f-45fd-b79c-50f9ca6c7da3/volumes"
Nov 04 01:56:11.832937 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:11.832899    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:56:11.992366 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:56:11.992327    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:56:38.782010 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.781974    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn"]
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782519    6116 cpu_manager.go:414] "containerMap: removing container" podUID="02295850-87a3-45a0-aa2e-1f42985f6c79" containerName="payload-pre-processing"
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782533    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="02295850-87a3-45a0-aa2e-1f42985f6c79" containerName="payload-pre-processing"
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782550    6116 cpu_manager.go:414] "containerMap: removing container" podUID="35deac99-14d0-416d-9c8d-c8abb03f5f7d" containerName="istio-proxy"
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782555    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35deac99-14d0-416d-9c8d-c8abb03f5f7d" containerName="istio-proxy"
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782573    6116 cpu_manager.go:414] "containerMap: removing container" podUID="b42d0649-0316-4a47-af9f-e271c5b7068f" containerName="main"
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782579    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b42d0649-0316-4a47-af9f-e271c5b7068f" containerName="main"
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782586    6116 cpu_manager.go:414] "containerMap: removing container" podUID="5e55d400-e27f-45fd-b79c-50f9ca6c7da3" containerName="istio-proxy"
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782591    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e55d400-e27f-45fd-b79c-50f9ca6c7da3" containerName="istio-proxy"
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782604    6116 cpu_manager.go:414] "containerMap: removing container" podUID="fee5848c-8694-4d7e-96d9-7536e13022a3" containerName="payload-processing"
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782609    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fee5848c-8694-4d7e-96d9-7536e13022a3" containerName="payload-processing"
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782618    6116 cpu_manager.go:414] "containerMap: removing container" podUID="33bf4adf-572f-4369-9f99-b685fb65f9e0" containerName="revoke-keys"
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782623    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33bf4adf-572f-4369-9f99-b685fb65f9e0" containerName="revoke-keys"
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782711    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="33bf4adf-572f-4369-9f99-b685fb65f9e0" containerName="revoke-keys"
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782722    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="35deac99-14d0-416d-9c8d-c8abb03f5f7d" containerName="istio-proxy"
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782729    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="02295850-87a3-45a0-aa2e-1f42985f6c79" containerName="payload-pre-processing"
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782736    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="35deac99-14d0-416d-9c8d-c8abb03f5f7d" containerName="istio-proxy"
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782742    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="b42d0649-0316-4a47-af9f-e271c5b7068f" containerName="main"
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782747    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e55d400-e27f-45fd-b79c-50f9ca6c7da3" containerName="istio-proxy"
Nov 04 01:56:38.782758 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.782753    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="fee5848c-8694-4d7e-96d9-7536e13022a3" containerName="payload-processing"
Nov 04 01:56:38.785038 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.785016    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn"
Nov 04 01:56:38.839183 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.839150    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 01:56:38.852670 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.852635    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/044a1757-03c3-4964-a3ab-71c685253791-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-acfc6aca-5bwkn\" (UID: \"044a1757-03c3-4964-a3ab-71c685253791\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn"
Nov 04 01:56:38.852867 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.852688    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-825w8\" (UniqueName: \"kubernetes.io/projected/044a1757-03c3-4964-a3ab-71c685253791-kube-api-access-825w8\") pod \"maas-api-revoke-sub-acfc6aca-5bwkn\" (UID: \"044a1757-03c3-4964-a3ab-71c685253791\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn"
Nov 04 01:56:38.852934 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.852868    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/044a1757-03c3-4964-a3ab-71c685253791-maas-api-xks-ca\") pod \"maas-api-revoke-sub-acfc6aca-5bwkn\" (UID: \"044a1757-03c3-4964-a3ab-71c685253791\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn"
Nov 04 01:56:38.888664 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.888627    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:56:38.888664 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.888641    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bj8b6\""
Nov 04 01:56:38.888899 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.888667    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:56:38.892783 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.892759    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn"]
Nov 04 01:56:38.953691 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.953660    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/044a1757-03c3-4964-a3ab-71c685253791-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-acfc6aca-5bwkn\" (UID: \"044a1757-03c3-4964-a3ab-71c685253791\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn"
Nov 04 01:56:38.953861 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.953703    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-825w8\" (UniqueName: \"kubernetes.io/projected/044a1757-03c3-4964-a3ab-71c685253791-kube-api-access-825w8\") pod \"maas-api-revoke-sub-acfc6aca-5bwkn\" (UID: \"044a1757-03c3-4964-a3ab-71c685253791\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn"
Nov 04 01:56:38.953861 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.953773    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/044a1757-03c3-4964-a3ab-71c685253791-maas-api-xks-ca\") pod \"maas-api-revoke-sub-acfc6aca-5bwkn\" (UID: \"044a1757-03c3-4964-a3ab-71c685253791\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn"
Nov 04 01:56:38.954563 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.954406    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/044a1757-03c3-4964-a3ab-71c685253791-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-acfc6aca-5bwkn\" (UID: \"044a1757-03c3-4964-a3ab-71c685253791\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn"
Nov 04 01:56:38.956263 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:38.956234    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/044a1757-03c3-4964-a3ab-71c685253791-maas-api-xks-ca\") pod \"maas-api-revoke-sub-acfc6aca-5bwkn\" (UID: \"044a1757-03c3-4964-a3ab-71c685253791\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn"
Nov 04 01:56:39.015215 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:39.015181    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-825w8\" (UniqueName: \"kubernetes.io/projected/044a1757-03c3-4964-a3ab-71c685253791-kube-api-access-825w8\") pod \"maas-api-revoke-sub-acfc6aca-5bwkn\" (UID: \"044a1757-03c3-4964-a3ab-71c685253791\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn"
Nov 04 01:56:39.099469 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:39.099365    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn"
Nov 04 01:56:39.473734 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:39.473704    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn"]
Nov 04 01:56:39.475140 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:56:39.475115    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod044a1757_03c3_4964_a3ab_71c685253791.slice/crio-caa2ca6660276da34e667c5eb760254cd4c1586731ac10851f4ebb452046d884 WatchSource:0}: Error finding container caa2ca6660276da34e667c5eb760254cd4c1586731ac10851f4ebb452046d884: Status 404 returned error can't find the container with id caa2ca6660276da34e667c5eb760254cd4c1586731ac10851f4ebb452046d884
Nov 04 01:56:39.903391 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:39.903357    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn" event={"ID":"044a1757-03c3-4964-a3ab-71c685253791","Type":"ContainerStarted","Data":"fbbb7af8a3d87c38db04d8932cdd39bb4943d326c6aeccd040d87620408b3872"}
Nov 04 01:56:39.903801 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:39.903417    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn" event={"ID":"044a1757-03c3-4964-a3ab-71c685253791","Type":"ContainerStarted","Data":"caa2ca6660276da34e667c5eb760254cd4c1586731ac10851f4ebb452046d884"}
Nov 04 01:56:40.019830 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:40.019724    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn" podStartSLOduration=2.019709286 podStartE2EDuration="2.019709286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:56:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:40.017412499 +0000 UTC m=+1285.138690079" watchObservedRunningTime="2026-11-04 01:56:40.019709286 +0000 UTC m=+1285.140986836"
Nov 04 01:56:40.911559 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:40.911526    6116 generic.go:356] "Generic (PLEG): container finished" podID="044a1757-03c3-4964-a3ab-71c685253791" containerID="fbbb7af8a3d87c38db04d8932cdd39bb4943d326c6aeccd040d87620408b3872" exitCode=0
Nov 04 01:56:40.911945 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:40.911595    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn" event={"ID":"044a1757-03c3-4964-a3ab-71c685253791","Type":"ContainerDied","Data":"fbbb7af8a3d87c38db04d8932cdd39bb4943d326c6aeccd040d87620408b3872"}
Nov 04 01:56:41.999233 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:56:41.999187    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:56:42.057327 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:42.057298    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn"
Nov 04 01:56:42.087603 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:42.087573    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/044a1757-03c3-4964-a3ab-71c685253791-kube-api-access-825w8\" (UniqueName: \"kubernetes.io/projected/044a1757-03c3-4964-a3ab-71c685253791-kube-api-access-825w8\") pod \"044a1757-03c3-4964-a3ab-71c685253791\" (UID: \"044a1757-03c3-4964-a3ab-71c685253791\") "
Nov 04 01:56:42.087723 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:42.087652    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/044a1757-03c3-4964-a3ab-71c685253791-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/044a1757-03c3-4964-a3ab-71c685253791-maas-api-openshift-ca\") pod \"044a1757-03c3-4964-a3ab-71c685253791\" (UID: \"044a1757-03c3-4964-a3ab-71c685253791\") "
Nov 04 01:56:42.087783 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:42.087771    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/044a1757-03c3-4964-a3ab-71c685253791-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/044a1757-03c3-4964-a3ab-71c685253791-maas-api-xks-ca\") pod \"044a1757-03c3-4964-a3ab-71c685253791\" (UID: \"044a1757-03c3-4964-a3ab-71c685253791\") "
Nov 04 01:56:42.087994 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:42.087973    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/044a1757-03c3-4964-a3ab-71c685253791-maas-api-openshift-ca" pod "044a1757-03c3-4964-a3ab-71c685253791" (UID: "044a1757-03c3-4964-a3ab-71c685253791"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:42.088213 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:42.088187    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/044a1757-03c3-4964-a3ab-71c685253791-maas-api-openshift-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:42.089865 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:42.089842    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/044a1757-03c3-4964-a3ab-71c685253791-maas-api-xks-ca" pod "044a1757-03c3-4964-a3ab-71c685253791" (UID: "044a1757-03c3-4964-a3ab-71c685253791"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:56:42.089953 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:42.089844    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/044a1757-03c3-4964-a3ab-71c685253791-kube-api-access-825w8" pod "044a1757-03c3-4964-a3ab-71c685253791" (UID: "044a1757-03c3-4964-a3ab-71c685253791"). InnerVolumeSpecName "kube-api-access-825w8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:42.189734 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:42.189660    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/044a1757-03c3-4964-a3ab-71c685253791-maas-api-xks-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:42.189734 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:42.189693    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-825w8\" (UniqueName: \"kubernetes.io/projected/044a1757-03c3-4964-a3ab-71c685253791-kube-api-access-825w8\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:42.926408 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:42.926361    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn"
Nov 04 01:56:42.926748 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:42.926525    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn" event={"ID":"044a1757-03c3-4964-a3ab-71c685253791","Type":"ContainerDied","Data":"caa2ca6660276da34e667c5eb760254cd4c1586731ac10851f4ebb452046d884"}
Nov 04 01:56:42.926748 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:42.926582    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="caa2ca6660276da34e667c5eb760254cd4c1586731ac10851f4ebb452046d884"
Nov 04 01:56:42.927784 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:42.927752    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"]
Nov 04 01:56:42.938861 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:42.938836    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"]
Nov 04 01:56:42.943285 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:42.941822    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"]
Nov 04 01:56:43.043891 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:43.043857    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:56:46.682075 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.682042    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz"]
Nov 04 01:56:46.682597 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.682583    6116 cpu_manager.go:414] "containerMap: removing container" podUID="35deac99-14d0-416d-9c8d-c8abb03f5f7d" containerName="istio-proxy"
Nov 04 01:56:46.682651 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.682598    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35deac99-14d0-416d-9c8d-c8abb03f5f7d" containerName="istio-proxy"
Nov 04 01:56:46.682651 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.682627    6116 cpu_manager.go:414] "containerMap: removing container" podUID="044a1757-03c3-4964-a3ab-71c685253791" containerName="revoke-keys"
Nov 04 01:56:46.682651 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.682633    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="044a1757-03c3-4964-a3ab-71c685253791" containerName="revoke-keys"
Nov 04 01:56:46.682745 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.682714    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="044a1757-03c3-4964-a3ab-71c685253791" containerName="revoke-keys"
Nov 04 01:56:46.684879 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.684863    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz"
Nov 04 01:56:46.764013 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.763969    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/654a6571-b6fd-40f0-a369-87a487f1b508-maas-api-xks-ca\") pod \"maas-api-revoke-sub-287cf977-rk7mz\" (UID: \"654a6571-b6fd-40f0-a369-87a487f1b508\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz"
Nov 04 01:56:46.764197 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.764027    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/654a6571-b6fd-40f0-a369-87a487f1b508-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-287cf977-rk7mz\" (UID: \"654a6571-b6fd-40f0-a369-87a487f1b508\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz"
Nov 04 01:56:46.764197 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.764145    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94gw5\" (UniqueName: \"kubernetes.io/projected/654a6571-b6fd-40f0-a369-87a487f1b508-kube-api-access-94gw5\") pod \"maas-api-revoke-sub-287cf977-rk7mz\" (UID: \"654a6571-b6fd-40f0-a369-87a487f1b508\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz"
Nov 04 01:56:46.786409 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.786376    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:56:46.786606 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.786411    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 01:56:46.786606 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.786527    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bj8b6\""
Nov 04 01:56:46.786606 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.786590    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:56:46.789641 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.789619    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz"]
Nov 04 01:56:46.865514 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.865438    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/654a6571-b6fd-40f0-a369-87a487f1b508-maas-api-xks-ca\") pod \"maas-api-revoke-sub-287cf977-rk7mz\" (UID: \"654a6571-b6fd-40f0-a369-87a487f1b508\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz"
Nov 04 01:56:46.865717 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.865522    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/654a6571-b6fd-40f0-a369-87a487f1b508-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-287cf977-rk7mz\" (UID: \"654a6571-b6fd-40f0-a369-87a487f1b508\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz"
Nov 04 01:56:46.865717 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.865572    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94gw5\" (UniqueName: \"kubernetes.io/projected/654a6571-b6fd-40f0-a369-87a487f1b508-kube-api-access-94gw5\") pod \"maas-api-revoke-sub-287cf977-rk7mz\" (UID: \"654a6571-b6fd-40f0-a369-87a487f1b508\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz"
Nov 04 01:56:46.866192 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.866174    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/654a6571-b6fd-40f0-a369-87a487f1b508-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-287cf977-rk7mz\" (UID: \"654a6571-b6fd-40f0-a369-87a487f1b508\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz"
Nov 04 01:56:46.867959 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.867937    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/654a6571-b6fd-40f0-a369-87a487f1b508-maas-api-xks-ca\") pod \"maas-api-revoke-sub-287cf977-rk7mz\" (UID: \"654a6571-b6fd-40f0-a369-87a487f1b508\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz"
Nov 04 01:56:46.926305 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.926274    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94gw5\" (UniqueName: \"kubernetes.io/projected/654a6571-b6fd-40f0-a369-87a487f1b508-kube-api-access-94gw5\") pod \"maas-api-revoke-sub-287cf977-rk7mz\" (UID: \"654a6571-b6fd-40f0-a369-87a487f1b508\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz"
Nov 04 01:56:46.997236 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:46.997152    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz"
Nov 04 01:56:47.370639 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:47.370615    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz"]
Nov 04 01:56:47.371944 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:56:47.371915    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod654a6571_b6fd_40f0_a369_87a487f1b508.slice/crio-d7f91628b71a6d1a787f17eca618e41635599923baf1a1c265dda350e4258dfb WatchSource:0}: Error finding container d7f91628b71a6d1a787f17eca618e41635599923baf1a1c265dda350e4258dfb: Status 404 returned error can't find the container with id d7f91628b71a6d1a787f17eca618e41635599923baf1a1c265dda350e4258dfb
Nov 04 01:56:47.965372 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:47.965338    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz" event={"ID":"654a6571-b6fd-40f0-a369-87a487f1b508","Type":"ContainerStarted","Data":"61d818ec90fc1d1c34d9f0a44a288313065113c905cb53963bc84e1a746cad08"}
Nov 04 01:56:47.965770 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:47.965388    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz" event={"ID":"654a6571-b6fd-40f0-a369-87a487f1b508","Type":"ContainerStarted","Data":"d7f91628b71a6d1a787f17eca618e41635599923baf1a1c265dda350e4258dfb"}
Nov 04 01:56:48.083898 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:48.083801    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz" podStartSLOduration=2.083786885 podStartE2EDuration="2.083786885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:56:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:48.080318129 +0000 UTC m=+1293.201595666" watchObservedRunningTime="2026-11-04 01:56:48.083786885 +0000 UTC m=+1293.205064411"
Nov 04 01:56:48.973237 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:48.973200    6116 generic.go:356] "Generic (PLEG): container finished" podID="654a6571-b6fd-40f0-a369-87a487f1b508" containerID="61d818ec90fc1d1c34d9f0a44a288313065113c905cb53963bc84e1a746cad08" exitCode=0
Nov 04 01:56:48.973661 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:48.973283    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz" event={"ID":"654a6571-b6fd-40f0-a369-87a487f1b508","Type":"ContainerDied","Data":"61d818ec90fc1d1c34d9f0a44a288313065113c905cb53963bc84e1a746cad08"}
Nov 04 01:56:50.124890 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.124866    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz"
Nov 04 01:56:50.206677 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.206647    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/654a6571-b6fd-40f0-a369-87a487f1b508-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/654a6571-b6fd-40f0-a369-87a487f1b508-maas-api-xks-ca\") pod \"654a6571-b6fd-40f0-a369-87a487f1b508\" (UID: \"654a6571-b6fd-40f0-a369-87a487f1b508\") "
Nov 04 01:56:50.206841 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.206722    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/654a6571-b6fd-40f0-a369-87a487f1b508-kube-api-access-94gw5\" (UniqueName: \"kubernetes.io/projected/654a6571-b6fd-40f0-a369-87a487f1b508-kube-api-access-94gw5\") pod \"654a6571-b6fd-40f0-a369-87a487f1b508\" (UID: \"654a6571-b6fd-40f0-a369-87a487f1b508\") "
Nov 04 01:56:50.206841 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.206780    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/654a6571-b6fd-40f0-a369-87a487f1b508-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/654a6571-b6fd-40f0-a369-87a487f1b508-maas-api-openshift-ca\") pod \"654a6571-b6fd-40f0-a369-87a487f1b508\" (UID: \"654a6571-b6fd-40f0-a369-87a487f1b508\") "
Nov 04 01:56:50.207179 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.207156    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/654a6571-b6fd-40f0-a369-87a487f1b508-maas-api-openshift-ca" pod "654a6571-b6fd-40f0-a369-87a487f1b508" (UID: "654a6571-b6fd-40f0-a369-87a487f1b508"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:50.207429 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.207325    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/654a6571-b6fd-40f0-a369-87a487f1b508-maas-api-openshift-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:50.208879 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.208845    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/654a6571-b6fd-40f0-a369-87a487f1b508-maas-api-xks-ca" pod "654a6571-b6fd-40f0-a369-87a487f1b508" (UID: "654a6571-b6fd-40f0-a369-87a487f1b508"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:56:50.208983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.208888    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/654a6571-b6fd-40f0-a369-87a487f1b508-kube-api-access-94gw5" pod "654a6571-b6fd-40f0-a369-87a487f1b508" (UID: "654a6571-b6fd-40f0-a369-87a487f1b508"). InnerVolumeSpecName "kube-api-access-94gw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:50.240351 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.240286    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf"]
Nov 04 01:56:50.240887 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.240872    6116 cpu_manager.go:414] "containerMap: removing container" podUID="654a6571-b6fd-40f0-a369-87a487f1b508" containerName="revoke-keys"
Nov 04 01:56:50.240934 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.240890    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="654a6571-b6fd-40f0-a369-87a487f1b508" containerName="revoke-keys"
Nov 04 01:56:50.240969 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.240966    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="654a6571-b6fd-40f0-a369-87a487f1b508" containerName="revoke-keys"
Nov 04 01:56:50.243096 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.243081    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf"
Nov 04 01:56:50.302907 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.302875    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf"]
Nov 04 01:56:50.309184 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.309152    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgwvz\" (UniqueName: \"kubernetes.io/projected/863ad259-3b19-4a29-bacf-5a6c8bc2222c-kube-api-access-wgwvz\") pod \"maas-api-revoke-sub-69b62d78-f4xgf\" (UID: \"863ad259-3b19-4a29-bacf-5a6c8bc2222c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf"
Nov 04 01:56:50.309328 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.309290    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/863ad259-3b19-4a29-bacf-5a6c8bc2222c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-69b62d78-f4xgf\" (UID: \"863ad259-3b19-4a29-bacf-5a6c8bc2222c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf"
Nov 04 01:56:50.309403 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.309387    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/863ad259-3b19-4a29-bacf-5a6c8bc2222c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-69b62d78-f4xgf\" (UID: \"863ad259-3b19-4a29-bacf-5a6c8bc2222c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf"
Nov 04 01:56:50.309565 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.309550    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/654a6571-b6fd-40f0-a369-87a487f1b508-maas-api-xks-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:50.309613 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.309571    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94gw5\" (UniqueName: \"kubernetes.io/projected/654a6571-b6fd-40f0-a369-87a487f1b508-kube-api-access-94gw5\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:50.410140 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.410110    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/863ad259-3b19-4a29-bacf-5a6c8bc2222c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-69b62d78-f4xgf\" (UID: \"863ad259-3b19-4a29-bacf-5a6c8bc2222c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf"
Nov 04 01:56:50.410340 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.410198    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/863ad259-3b19-4a29-bacf-5a6c8bc2222c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-69b62d78-f4xgf\" (UID: \"863ad259-3b19-4a29-bacf-5a6c8bc2222c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf"
Nov 04 01:56:50.410340 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.410300    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgwvz\" (UniqueName: \"kubernetes.io/projected/863ad259-3b19-4a29-bacf-5a6c8bc2222c-kube-api-access-wgwvz\") pod \"maas-api-revoke-sub-69b62d78-f4xgf\" (UID: \"863ad259-3b19-4a29-bacf-5a6c8bc2222c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf"
Nov 04 01:56:50.410851 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.410826    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/863ad259-3b19-4a29-bacf-5a6c8bc2222c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-69b62d78-f4xgf\" (UID: \"863ad259-3b19-4a29-bacf-5a6c8bc2222c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf"
Nov 04 01:56:50.412722 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.412699    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/863ad259-3b19-4a29-bacf-5a6c8bc2222c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-69b62d78-f4xgf\" (UID: \"863ad259-3b19-4a29-bacf-5a6c8bc2222c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf"
Nov 04 01:56:50.478199 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.478163    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgwvz\" (UniqueName: \"kubernetes.io/projected/863ad259-3b19-4a29-bacf-5a6c8bc2222c-kube-api-access-wgwvz\") pod \"maas-api-revoke-sub-69b62d78-f4xgf\" (UID: \"863ad259-3b19-4a29-bacf-5a6c8bc2222c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf"
Nov 04 01:56:50.554183 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.554107    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf"
Nov 04 01:56:50.934765 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.934735    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf"]
Nov 04 01:56:50.936317 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:56:50.936289    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod863ad259_3b19_4a29_bacf_5a6c8bc2222c.slice/crio-bc92f2ce5e3249b364ce51e49e3632f137d8e84076c6fcebf766040bfd8edf21 WatchSource:0}: Error finding container bc92f2ce5e3249b364ce51e49e3632f137d8e84076c6fcebf766040bfd8edf21: Status 404 returned error can't find the container with id bc92f2ce5e3249b364ce51e49e3632f137d8e84076c6fcebf766040bfd8edf21
Nov 04 01:56:50.986783 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.986753    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf" event={"ID":"863ad259-3b19-4a29-bacf-5a6c8bc2222c","Type":"ContainerStarted","Data":"bc92f2ce5e3249b364ce51e49e3632f137d8e84076c6fcebf766040bfd8edf21"}
Nov 04 01:56:50.988107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.988082    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz" event={"ID":"654a6571-b6fd-40f0-a369-87a487f1b508","Type":"ContainerDied","Data":"d7f91628b71a6d1a787f17eca618e41635599923baf1a1c265dda350e4258dfb"}
Nov 04 01:56:50.988218 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.988127    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7f91628b71a6d1a787f17eca618e41635599923baf1a1c265dda350e4258dfb"
Nov 04 01:56:50.988218 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:50.988087    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz"
Nov 04 01:56:51.996311 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:51.996275    6116 generic.go:356] "Generic (PLEG): container finished" podID="863ad259-3b19-4a29-bacf-5a6c8bc2222c" containerID="69e1753da3ee6fd06e220849990fe60004cae25a99bb9cb0c07138391d12a3c1" exitCode=0
Nov 04 01:56:51.996741 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:51.996359    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf" event={"ID":"863ad259-3b19-4a29-bacf-5a6c8bc2222c","Type":"ContainerDied","Data":"69e1753da3ee6fd06e220849990fe60004cae25a99bb9cb0c07138391d12a3c1"}
Nov 04 01:56:53.144190 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:53.144167    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf"
Nov 04 01:56:53.242759 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:53.242727    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/863ad259-3b19-4a29-bacf-5a6c8bc2222c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/863ad259-3b19-4a29-bacf-5a6c8bc2222c-maas-api-openshift-ca\") pod \"863ad259-3b19-4a29-bacf-5a6c8bc2222c\" (UID: \"863ad259-3b19-4a29-bacf-5a6c8bc2222c\") "
Nov 04 01:56:53.242959 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:53.242870    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/863ad259-3b19-4a29-bacf-5a6c8bc2222c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/863ad259-3b19-4a29-bacf-5a6c8bc2222c-maas-api-xks-ca\") pod \"863ad259-3b19-4a29-bacf-5a6c8bc2222c\" (UID: \"863ad259-3b19-4a29-bacf-5a6c8bc2222c\") "
Nov 04 01:56:53.243020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:53.242988    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/863ad259-3b19-4a29-bacf-5a6c8bc2222c-kube-api-access-wgwvz\" (UniqueName: \"kubernetes.io/projected/863ad259-3b19-4a29-bacf-5a6c8bc2222c-kube-api-access-wgwvz\") pod \"863ad259-3b19-4a29-bacf-5a6c8bc2222c\" (UID: \"863ad259-3b19-4a29-bacf-5a6c8bc2222c\") "
Nov 04 01:56:53.243171 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:53.243144    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/863ad259-3b19-4a29-bacf-5a6c8bc2222c-maas-api-openshift-ca" pod "863ad259-3b19-4a29-bacf-5a6c8bc2222c" (UID: "863ad259-3b19-4a29-bacf-5a6c8bc2222c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:53.243495 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:53.243472    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/863ad259-3b19-4a29-bacf-5a6c8bc2222c-maas-api-openshift-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:53.245155 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:53.245135    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/863ad259-3b19-4a29-bacf-5a6c8bc2222c-maas-api-xks-ca" pod "863ad259-3b19-4a29-bacf-5a6c8bc2222c" (UID: "863ad259-3b19-4a29-bacf-5a6c8bc2222c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:56:53.245155 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:53.245139    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/863ad259-3b19-4a29-bacf-5a6c8bc2222c-kube-api-access-wgwvz" pod "863ad259-3b19-4a29-bacf-5a6c8bc2222c" (UID: "863ad259-3b19-4a29-bacf-5a6c8bc2222c"). InnerVolumeSpecName "kube-api-access-wgwvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:53.344763 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:53.344673    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/863ad259-3b19-4a29-bacf-5a6c8bc2222c-maas-api-xks-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:53.344763 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:53.344698    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgwvz\" (UniqueName: \"kubernetes.io/projected/863ad259-3b19-4a29-bacf-5a6c8bc2222c-kube-api-access-wgwvz\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:53.809230 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:53.809194    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v"]
Nov 04 01:56:53.906386 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:53.906347    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4"]
Nov 04 01:56:54.010617 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:54.010585    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf"
Nov 04 01:56:54.010799 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:54.010613    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-69b62d78-f4xgf" event={"ID":"863ad259-3b19-4a29-bacf-5a6c8bc2222c","Type":"ContainerDied","Data":"bc92f2ce5e3249b364ce51e49e3632f137d8e84076c6fcebf766040bfd8edf21"}
Nov 04 01:56:54.010799 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:54.010682    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc92f2ce5e3249b364ce51e49e3632f137d8e84076c6fcebf766040bfd8edf21"
Nov 04 01:56:57.953275 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:57.953212    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk" podUID="a9f371eb-274b-42c2-ac99-83ba56e24600" containerName="main" containerID="cri-o://0d0f3e0fa5adff59bb5e810d90633148ad38b785884188dc0fa9f5a33d643a69" gracePeriod=45
Nov 04 01:56:57.968911 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:57.968875    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr" podUID="2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7" containerName="main" containerID="cri-o://ad52477463364614b7653bd9dde4b90ea67fe644b8f66e8de94d61f2dbfa94cf" gracePeriod=45
Nov 04 01:56:57.969012 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:57.968890    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf" podUID="e1c8cce3-72a2-4d03-8680-42e8cd439f3b" containerName="main" containerID="cri-o://28a3fe6fc50610d5a47e2ebedfb5742454c171e43c57f1af4fdff3186b228ddb" gracePeriod=45
Nov 04 01:56:58.263438 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.263412    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:56:58.315932 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.315897    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:56:58.319549 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.319527    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:56:58.403307 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.403274    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-model-cache\") pod \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") "
Nov 04 01:56:58.403307 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.403322    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-tls-certs\" (UniqueName: \"kubernetes.io/secret/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-tls-certs\") pod \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") "
Nov 04 01:56:58.403307 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.403378    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a9f371eb-274b-42c2-ac99-83ba56e24600-tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f371eb-274b-42c2-ac99-83ba56e24600-tls-certs\") pod \"a9f371eb-274b-42c2-ac99-83ba56e24600\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") "
Nov 04 01:56:58.403307 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.403416    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-model-cache\") pod \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") "
Nov 04 01:56:58.403307 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.403476    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-home\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-home\") pod \"a9f371eb-274b-42c2-ac99-83ba56e24600\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") "
Nov 04 01:56:58.403942 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.403516    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9f371eb-274b-42c2-ac99-83ba56e24600-kube-api-access-8prbq\" (UniqueName: \"kubernetes.io/projected/a9f371eb-274b-42c2-ac99-83ba56e24600-kube-api-access-8prbq\") pod \"a9f371eb-274b-42c2-ac99-83ba56e24600\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") "
Nov 04 01:56:58.403942 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.403565    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-dshm\") pod \"a9f371eb-274b-42c2-ac99-83ba56e24600\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") "
Nov 04 01:56:58.403942 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.403617    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-model-cache" pod "2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7" (UID: "2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:58.403942 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.403624    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-home\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-home\") pod \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") "
Nov 04 01:56:58.403942 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.403650    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-kube-api-access-kgrk2\" (UniqueName: \"kubernetes.io/projected/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-kube-api-access-kgrk2\") pod \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") "
Nov 04 01:56:58.403942 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.403679    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-tmp-dir\") pod \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") "
Nov 04 01:56:58.403942 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.403754    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-kube-api-access-qdlx9\" (UniqueName: \"kubernetes.io/projected/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-kube-api-access-qdlx9\") pod \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") "
Nov 04 01:56:58.403942 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.403784    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-dshm\") pod \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") "
Nov 04 01:56:58.403942 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.403810    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-model-cache\") pod \"a9f371eb-274b-42c2-ac99-83ba56e24600\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") "
Nov 04 01:56:58.403942 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.403844    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-model-cache" pod "e1c8cce3-72a2-4d03-8680-42e8cd439f3b" (UID: "e1c8cce3-72a2-4d03-8680-42e8cd439f3b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:58.404499 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.404051    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-model-cache" pod "a9f371eb-274b-42c2-ac99-83ba56e24600" (UID: "a9f371eb-274b-42c2-ac99-83ba56e24600"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:58.404499 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.404216    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-home" pod "a9f371eb-274b-42c2-ac99-83ba56e24600" (UID: "a9f371eb-274b-42c2-ac99-83ba56e24600"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:58.404597 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.404512    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-dshm\") pod \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") "
Nov 04 01:56:58.404643 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.404610    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-tmp-dir\") pod \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") "
Nov 04 01:56:58.404702 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.404661    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-tls-certs\" (UniqueName: \"kubernetes.io/secret/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-tls-certs\") pod \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\" (UID: \"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7\") "
Nov 04 01:56:58.404702 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.404690    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-home\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-home\") pod \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\" (UID: \"e1c8cce3-72a2-4d03-8680-42e8cd439f3b\") "
Nov 04 01:56:58.404816 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.404754    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-tmp-dir\") pod \"a9f371eb-274b-42c2-ac99-83ba56e24600\" (UID: \"a9f371eb-274b-42c2-ac99-83ba56e24600\") "
Nov 04 01:56:58.405664 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.405206    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-tmp-dir" pod "e1c8cce3-72a2-4d03-8680-42e8cd439f3b" (UID: "e1c8cce3-72a2-4d03-8680-42e8cd439f3b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:58.405664 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.405510    6116 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-model-cache\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.405664 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.405531    6116 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-tmp-dir\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.405664 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.405547    6116 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-model-cache\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.405664 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.405560    6116 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-model-cache\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.405664 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.405571    6116 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-home\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.406084 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.405774    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-tmp-dir" pod "a9f371eb-274b-42c2-ac99-83ba56e24600" (UID: "a9f371eb-274b-42c2-ac99-83ba56e24600"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:58.406084 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.405963    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-home" pod "e1c8cce3-72a2-4d03-8680-42e8cd439f3b" (UID: "e1c8cce3-72a2-4d03-8680-42e8cd439f3b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:58.407021 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.406760    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-tmp-dir" pod "2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7" (UID: "2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:58.407021 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.406966    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-home" pod "2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7" (UID: "2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:58.411386 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.410790    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-dshm" pod "e1c8cce3-72a2-4d03-8680-42e8cd439f3b" (UID: "e1c8cce3-72a2-4d03-8680-42e8cd439f3b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:58.411386 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.411259    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-tls-certs" pod "2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7" (UID: "2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:56:58.411386 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.411272    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-dshm" pod "2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7" (UID: "2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:58.411386 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.411311    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-kube-api-access-kgrk2" pod "2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7" (UID: "2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7"). InnerVolumeSpecName "kube-api-access-kgrk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:58.411386 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.411422    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-tls-certs" pod "e1c8cce3-72a2-4d03-8680-42e8cd439f3b" (UID: "e1c8cce3-72a2-4d03-8680-42e8cd439f3b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:56:58.412636 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.411433    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9f371eb-274b-42c2-ac99-83ba56e24600-tls-certs" pod "a9f371eb-274b-42c2-ac99-83ba56e24600" (UID: "a9f371eb-274b-42c2-ac99-83ba56e24600"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:56:58.412636 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.411518    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-kube-api-access-qdlx9" pod "e1c8cce3-72a2-4d03-8680-42e8cd439f3b" (UID: "e1c8cce3-72a2-4d03-8680-42e8cd439f3b"). InnerVolumeSpecName "kube-api-access-qdlx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:58.412636 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.412555    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9f371eb-274b-42c2-ac99-83ba56e24600-kube-api-access-8prbq" pod "a9f371eb-274b-42c2-ac99-83ba56e24600" (UID: "a9f371eb-274b-42c2-ac99-83ba56e24600"). InnerVolumeSpecName "kube-api-access-8prbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:58.413030 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.413006    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-dshm" pod "a9f371eb-274b-42c2-ac99-83ba56e24600" (UID: "a9f371eb-274b-42c2-ac99-83ba56e24600"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:56:58.507340 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.507259    6116 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-dshm\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.507340 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.507285    6116 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-dshm\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.507340 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.507299    6116 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-tls-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.507340 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.507309    6116 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-home\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.507340 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.507318    6116 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-tmp-dir\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.507340 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.507325    6116 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-tls-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.507340 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.507332    6116 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f371eb-274b-42c2-ac99-83ba56e24600-tls-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.507340 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.507341    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8prbq\" (UniqueName: \"kubernetes.io/projected/a9f371eb-274b-42c2-ac99-83ba56e24600-kube-api-access-8prbq\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.507880 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.507350    6116 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9f371eb-274b-42c2-ac99-83ba56e24600-dshm\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.507880 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.507359    6116 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-home\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.507880 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.507366    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgrk2\" (UniqueName: \"kubernetes.io/projected/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-kube-api-access-kgrk2\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.507880 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.507374    6116 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7-tmp-dir\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.507880 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.507383    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdlx9\" (UniqueName: \"kubernetes.io/projected/e1c8cce3-72a2-4d03-8680-42e8cd439f3b-kube-api-access-qdlx9\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:58.810406 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.810299    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v" podUID="7ba76e9b-c899-45e6-85a6-fcaffea854a4" containerName="payload-processing" containerID="cri-o://2919ca11159f88abdb64f533d45f5f52047689d56bf9b3b25cc05a77e579cf2b" gracePeriod=25
Nov 04 01:56:58.906675 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:58.906631    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4" podUID="79e5a497-975f-4f5f-b76f-20213e9f202d" containerName="payload-pre-processing" containerID="cri-o://b0b59c99f0b7ed51bb08fabd420a651e252b300d0c3c3b6dcf73479ac8740a37" gracePeriod=25
Nov 04 01:56:59.077748 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.077691    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v"
Nov 04 01:56:59.087653 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.087627    6116 generic.go:356] "Generic (PLEG): container finished" podID="e1c8cce3-72a2-4d03-8680-42e8cd439f3b" containerID="28a3fe6fc50610d5a47e2ebedfb5742454c171e43c57f1af4fdff3186b228ddb" exitCode=0
Nov 04 01:56:59.087774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.087671    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"
Nov 04 01:56:59.087774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.087710    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf" event={"ID":"e1c8cce3-72a2-4d03-8680-42e8cd439f3b","Type":"ContainerDied","Data":"28a3fe6fc50610d5a47e2ebedfb5742454c171e43c57f1af4fdff3186b228ddb"}
Nov 04 01:56:59.087774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.087754    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf" event={"ID":"e1c8cce3-72a2-4d03-8680-42e8cd439f3b","Type":"ContainerDied","Data":"17b82445b918af9f5b672b6d73e41f4d0eeac3efa3cafc2ab5ec9798730cd592"}
Nov 04 01:56:59.087774 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.087774    6116 scope.go:122] "RemoveContainer" containerID="28a3fe6fc50610d5a47e2ebedfb5742454c171e43c57f1af4fdff3186b228ddb"
Nov 04 01:56:59.089098 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.089071    6116 generic.go:356] "Generic (PLEG): container finished" podID="a9f371eb-274b-42c2-ac99-83ba56e24600" containerID="0d0f3e0fa5adff59bb5e810d90633148ad38b785884188dc0fa9f5a33d643a69" exitCode=0
Nov 04 01:56:59.089197 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.089098    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk" event={"ID":"a9f371eb-274b-42c2-ac99-83ba56e24600","Type":"ContainerDied","Data":"0d0f3e0fa5adff59bb5e810d90633148ad38b785884188dc0fa9f5a33d643a69"}
Nov 04 01:56:59.089197 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.089127    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"
Nov 04 01:56:59.089197 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.089145    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk" event={"ID":"a9f371eb-274b-42c2-ac99-83ba56e24600","Type":"ContainerDied","Data":"7b115b84b0a10fdb7ad756704855bd54f9ad62174aacfab7ba7f70958fcd6589"}
Nov 04 01:56:59.092689 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.092668    6116 generic.go:356] "Generic (PLEG): container finished" podID="7ba76e9b-c899-45e6-85a6-fcaffea854a4" containerID="2919ca11159f88abdb64f533d45f5f52047689d56bf9b3b25cc05a77e579cf2b" exitCode=0
Nov 04 01:56:59.092807 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.092713    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v"
Nov 04 01:56:59.092807 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.092745    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v" event={"ID":"7ba76e9b-c899-45e6-85a6-fcaffea854a4","Type":"ContainerDied","Data":"2919ca11159f88abdb64f533d45f5f52047689d56bf9b3b25cc05a77e579cf2b"}
Nov 04 01:56:59.092807 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.092798    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v" event={"ID":"7ba76e9b-c899-45e6-85a6-fcaffea854a4","Type":"ContainerDied","Data":"20b7346f3865b96648035dfbb39bb738a1be8b96f52a93182409812f2dfd36c2"}
Nov 04 01:56:59.094280 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.094260    6116 generic.go:356] "Generic (PLEG): container finished" podID="2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7" containerID="ad52477463364614b7653bd9dde4b90ea67fe644b8f66e8de94d61f2dbfa94cf" exitCode=0
Nov 04 01:56:59.094378 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.094286    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"
Nov 04 01:56:59.094378 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.094338    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr" event={"ID":"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7","Type":"ContainerDied","Data":"ad52477463364614b7653bd9dde4b90ea67fe644b8f66e8de94d61f2dbfa94cf"}
Nov 04 01:56:59.094378 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.094368    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr" event={"ID":"2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7","Type":"ContainerDied","Data":"6887f2852c73a9960725d3e5bfc91a24705098e853e5fd1d5f35d0195e71a479"}
Nov 04 01:56:59.098598 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.098576    6116 scope.go:122] "RemoveContainer" containerID="28a3fe6fc50610d5a47e2ebedfb5742454c171e43c57f1af4fdff3186b228ddb"
Nov 04 01:56:59.098865 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:56:59.098846    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28a3fe6fc50610d5a47e2ebedfb5742454c171e43c57f1af4fdff3186b228ddb\": container with ID starting with 28a3fe6fc50610d5a47e2ebedfb5742454c171e43c57f1af4fdff3186b228ddb not found: ID does not exist" containerID="28a3fe6fc50610d5a47e2ebedfb5742454c171e43c57f1af4fdff3186b228ddb"
Nov 04 01:56:59.098913 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.098871    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28a3fe6fc50610d5a47e2ebedfb5742454c171e43c57f1af4fdff3186b228ddb"} err="failed to get container status \"28a3fe6fc50610d5a47e2ebedfb5742454c171e43c57f1af4fdff3186b228ddb\": rpc error: code = NotFound desc = could not find container \"28a3fe6fc50610d5a47e2ebedfb5742454c171e43c57f1af4fdff3186b228ddb\": container with ID starting with 28a3fe6fc50610d5a47e2ebedfb5742454c171e43c57f1af4fdff3186b228ddb not found: ID does not exist"
Nov 04 01:56:59.098913 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.098889    6116 scope.go:122] "RemoveContainer" containerID="0d0f3e0fa5adff59bb5e810d90633148ad38b785884188dc0fa9f5a33d643a69"
Nov 04 01:56:59.107204 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.107190    6116 scope.go:122] "RemoveContainer" containerID="0d0f3e0fa5adff59bb5e810d90633148ad38b785884188dc0fa9f5a33d643a69"
Nov 04 01:56:59.107469 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:56:59.107436    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d0f3e0fa5adff59bb5e810d90633148ad38b785884188dc0fa9f5a33d643a69\": container with ID starting with 0d0f3e0fa5adff59bb5e810d90633148ad38b785884188dc0fa9f5a33d643a69 not found: ID does not exist" containerID="0d0f3e0fa5adff59bb5e810d90633148ad38b785884188dc0fa9f5a33d643a69"
Nov 04 01:56:59.107569 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.107479    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d0f3e0fa5adff59bb5e810d90633148ad38b785884188dc0fa9f5a33d643a69"} err="failed to get container status \"0d0f3e0fa5adff59bb5e810d90633148ad38b785884188dc0fa9f5a33d643a69\": rpc error: code = NotFound desc = could not find container \"0d0f3e0fa5adff59bb5e810d90633148ad38b785884188dc0fa9f5a33d643a69\": container with ID starting with 0d0f3e0fa5adff59bb5e810d90633148ad38b785884188dc0fa9f5a33d643a69 not found: ID does not exist"
Nov 04 01:56:59.107569 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.107495    6116 scope.go:122] "RemoveContainer" containerID="2919ca11159f88abdb64f533d45f5f52047689d56bf9b3b25cc05a77e579cf2b"
Nov 04 01:56:59.116005 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.115989    6116 scope.go:122] "RemoveContainer" containerID="2919ca11159f88abdb64f533d45f5f52047689d56bf9b3b25cc05a77e579cf2b"
Nov 04 01:56:59.116276 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:56:59.116253    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2919ca11159f88abdb64f533d45f5f52047689d56bf9b3b25cc05a77e579cf2b\": container with ID starting with 2919ca11159f88abdb64f533d45f5f52047689d56bf9b3b25cc05a77e579cf2b not found: ID does not exist" containerID="2919ca11159f88abdb64f533d45f5f52047689d56bf9b3b25cc05a77e579cf2b"
Nov 04 01:56:59.116332 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.116283    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2919ca11159f88abdb64f533d45f5f52047689d56bf9b3b25cc05a77e579cf2b"} err="failed to get container status \"2919ca11159f88abdb64f533d45f5f52047689d56bf9b3b25cc05a77e579cf2b\": rpc error: code = NotFound desc = could not find container \"2919ca11159f88abdb64f533d45f5f52047689d56bf9b3b25cc05a77e579cf2b\": container with ID starting with 2919ca11159f88abdb64f533d45f5f52047689d56bf9b3b25cc05a77e579cf2b not found: ID does not exist"
Nov 04 01:56:59.116332 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.116301    6116 scope.go:122] "RemoveContainer" containerID="ad52477463364614b7653bd9dde4b90ea67fe644b8f66e8de94d61f2dbfa94cf"
Nov 04 01:56:59.124804 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.124785    6116 scope.go:122] "RemoveContainer" containerID="ad52477463364614b7653bd9dde4b90ea67fe644b8f66e8de94d61f2dbfa94cf"
Nov 04 01:56:59.125076 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:56:59.125056    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad52477463364614b7653bd9dde4b90ea67fe644b8f66e8de94d61f2dbfa94cf\": container with ID starting with ad52477463364614b7653bd9dde4b90ea67fe644b8f66e8de94d61f2dbfa94cf not found: ID does not exist" containerID="ad52477463364614b7653bd9dde4b90ea67fe644b8f66e8de94d61f2dbfa94cf"
Nov 04 01:56:59.125139 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.125084    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad52477463364614b7653bd9dde4b90ea67fe644b8f66e8de94d61f2dbfa94cf"} err="failed to get container status \"ad52477463364614b7653bd9dde4b90ea67fe644b8f66e8de94d61f2dbfa94cf\": rpc error: code = NotFound desc = could not find container \"ad52477463364614b7653bd9dde4b90ea67fe644b8f66e8de94d61f2dbfa94cf\": container with ID starting with ad52477463364614b7653bd9dde4b90ea67fe644b8f66e8de94d61f2dbfa94cf not found: ID does not exist"
Nov 04 01:56:59.215376 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.215347    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ba76e9b-c899-45e6-85a6-fcaffea854a4-kube-api-access-v5xh7\" (UniqueName: \"kubernetes.io/projected/7ba76e9b-c899-45e6-85a6-fcaffea854a4-kube-api-access-v5xh7\") pod \"7ba76e9b-c899-45e6-85a6-fcaffea854a4\" (UID: \"7ba76e9b-c899-45e6-85a6-fcaffea854a4\") "
Nov 04 01:56:59.215591 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.215503    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ba76e9b-c899-45e6-85a6-fcaffea854a4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ba76e9b-c899-45e6-85a6-fcaffea854a4-plugins-config-volume\") pod \"7ba76e9b-c899-45e6-85a6-fcaffea854a4\" (UID: \"7ba76e9b-c899-45e6-85a6-fcaffea854a4\") "
Nov 04 01:56:59.215872 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.215844    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ba76e9b-c899-45e6-85a6-fcaffea854a4-plugins-config-volume" pod "7ba76e9b-c899-45e6-85a6-fcaffea854a4" (UID: "7ba76e9b-c899-45e6-85a6-fcaffea854a4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:56:59.215995 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.215973    6116 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ba76e9b-c899-45e6-85a6-fcaffea854a4-plugins-config-volume\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:59.217651 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.217627    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ba76e9b-c899-45e6-85a6-fcaffea854a4-kube-api-access-v5xh7" pod "7ba76e9b-c899-45e6-85a6-fcaffea854a4" (UID: "7ba76e9b-c899-45e6-85a6-fcaffea854a4"). InnerVolumeSpecName "kube-api-access-v5xh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:56:59.304410 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.304380    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"]
Nov 04 01:56:59.308939 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.308912    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/premium-w3-fa0149-kserve-5678946dbd-fpvmr"]
Nov 04 01:56:59.316929 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.316906    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5xh7\" (UniqueName: \"kubernetes.io/projected/7ba76e9b-c899-45e6-85a6-fcaffea854a4-kube-api-access-v5xh7\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:56:59.418389 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.418356    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"]
Nov 04 01:56:59.422291 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.422264    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/simulator-w3-fa0149-kserve-c5c64ccb5-k6fdf"]
Nov 04 01:56:59.530724 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.530687    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"]
Nov 04 01:56:59.540441 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.540411    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-fa0149/distinct-2-w3-fa0149-kserve-5f8cb9cdc6-q9qjk"]
Nov 04 01:56:59.701905 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.701873    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v"]
Nov 04 01:56:59.704564 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:56:59.704544    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v"]
Nov 04 01:57:01.029739 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:01.029697    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7" path="/var/lib/kubelet/pods/2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7/volumes"
Nov 04 01:57:01.030348 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:01.030327    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ba76e9b-c899-45e6-85a6-fcaffea854a4" path="/var/lib/kubelet/pods/7ba76e9b-c899-45e6-85a6-fcaffea854a4/volumes"
Nov 04 01:57:01.030998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:01.030976    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9f371eb-274b-42c2-ac99-83ba56e24600" path="/var/lib/kubelet/pods/a9f371eb-274b-42c2-ac99-83ba56e24600/volumes"
Nov 04 01:57:01.031984 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:01.031964    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1c8cce3-72a2-4d03-8680-42e8cd439f3b" path="/var/lib/kubelet/pods/e1c8cce3-72a2-4d03-8680-42e8cd439f3b/volumes"
Nov 04 01:57:06.010113 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:06.010071    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4" podUID="79e5a497-975f-4f5f-b76f-20213e9f202d" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:57:11.990946 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:57:11.990910    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:57:13.361340 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.361308    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"]
Nov 04 01:57:13.361869 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.361854    6116 cpu_manager.go:414] "containerMap: removing container" podUID="2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7" containerName="main"
Nov 04 01:57:13.361906 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.361870    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7" containerName="main"
Nov 04 01:57:13.361906 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.361884    6116 cpu_manager.go:414] "containerMap: removing container" podUID="863ad259-3b19-4a29-bacf-5a6c8bc2222c" containerName="revoke-keys"
Nov 04 01:57:13.361906 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.361889    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="863ad259-3b19-4a29-bacf-5a6c8bc2222c" containerName="revoke-keys"
Nov 04 01:57:13.361906 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.361899    6116 cpu_manager.go:414] "containerMap: removing container" podUID="a9f371eb-274b-42c2-ac99-83ba56e24600" containerName="main"
Nov 04 01:57:13.361906 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.361904    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9f371eb-274b-42c2-ac99-83ba56e24600" containerName="main"
Nov 04 01:57:13.362071 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.361925    6116 cpu_manager.go:414] "containerMap: removing container" podUID="7ba76e9b-c899-45e6-85a6-fcaffea854a4" containerName="payload-processing"
Nov 04 01:57:13.362071 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.361930    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ba76e9b-c899-45e6-85a6-fcaffea854a4" containerName="payload-processing"
Nov 04 01:57:13.362071 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.361936    6116 cpu_manager.go:414] "containerMap: removing container" podUID="e1c8cce3-72a2-4d03-8680-42e8cd439f3b" containerName="main"
Nov 04 01:57:13.362071 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.361939    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1c8cce3-72a2-4d03-8680-42e8cd439f3b" containerName="main"
Nov 04 01:57:13.362071 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.362011    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ba76e9b-c899-45e6-85a6-fcaffea854a4" containerName="payload-processing"
Nov 04 01:57:13.362071 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.362021    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="2fbf4e49-a222-4caf-b3d9-0fdbe663ebe7" containerName="main"
Nov 04 01:57:13.362071 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.362027    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="863ad259-3b19-4a29-bacf-5a6c8bc2222c" containerName="revoke-keys"
Nov 04 01:57:13.362071 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.362033    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9f371eb-274b-42c2-ac99-83ba56e24600" containerName="main"
Nov 04 01:57:13.362071 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.362040    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1c8cce3-72a2-4d03-8680-42e8cd439f3b" containerName="main"
Nov 04 01:57:13.366767 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.366746    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.383174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.383150    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-podinfo\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.383310 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.383193    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-token\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.383310 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.383291    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-data\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.383425 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.383377    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-envoy\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.383425 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.383407    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-workload-socket\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.383527 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.383504    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzlrq\" (UniqueName: \"kubernetes.io/projected/3d83b47f-120d-42b8-9e46-e7622dd29cfc-kube-api-access-jzlrq\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.383558 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.383550    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istiod-ca-cert\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.383646 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.383633    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-workload-certs\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.383679 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.383665    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-credential-socket\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.418892 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.418866    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-fa0149-openshift-default-dockercfg-9mtst\""
Nov 04 01:57:13.472632 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.472596    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"]
Nov 04 01:57:13.484483 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.484422    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-token\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.484671 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.484540    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-data\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.484671 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.484597    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-envoy\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.484671 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.484628    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-workload-socket\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.484836 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.484695    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzlrq\" (UniqueName: \"kubernetes.io/projected/3d83b47f-120d-42b8-9e46-e7622dd29cfc-kube-api-access-jzlrq\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.484836 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.484762    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istiod-ca-cert\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.484945 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.484831    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-workload-certs\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.484945 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.484870    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-credential-socket\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.484945 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.484894    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-podinfo\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.485082 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.485000    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-data\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.485082 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.485015    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-workload-socket\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.485569 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.485545    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-workload-certs\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.485739 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.485625    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-credential-socket\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.485879 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.485731    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istiod-ca-cert\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.487490 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.487427    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-envoy\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.487603 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.487565    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-podinfo\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.545464 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.545413    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-token\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.545800 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.545779    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzlrq\" (UniqueName: \"kubernetes.io/projected/3d83b47f-120d-42b8-9e46-e7622dd29cfc-kube-api-access-jzlrq\") pod \"e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") " pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:13.679460 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:13.679424    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:14.110416 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:14.110386    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"]
Nov 04 01:57:14.111873 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:57:14.111838    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d83b47f_120d_42b8_9e46_e7622dd29cfc.slice/crio-a29bcc1c36321f9105005f7b7201cc26a467a617af560e0c972dee1df9540016 WatchSource:0}: Error finding container a29bcc1c36321f9105005f7b7201cc26a467a617af560e0c972dee1df9540016: Status 404 returned error can't find the container with id a29bcc1c36321f9105005f7b7201cc26a467a617af560e0c972dee1df9540016
Nov 04 01:57:14.114071 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:14.114038    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:57:14.114149 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:14.114122    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:57:14.114189 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:14.114151    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:57:14.193134 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:14.193101    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn" event={"ID":"3d83b47f-120d-42b8-9e46-e7622dd29cfc","Type":"ContainerStarted","Data":"515727c60ae68944a2f959c648a3022295c14342034a50e62db0bd586939026f"}
Nov 04 01:57:14.193321 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:14.193165    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn" event={"ID":"3d83b47f-120d-42b8-9e46-e7622dd29cfc","Type":"ContainerStarted","Data":"a29bcc1c36321f9105005f7b7201cc26a467a617af560e0c972dee1df9540016"}
Nov 04 01:57:14.349775 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:14.349718    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn" podStartSLOduration=1.349703436 podStartE2EDuration="1.349703436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:57:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:57:14.348741359 +0000 UTC m=+1319.470018910" watchObservedRunningTime="2026-11-04 01:57:14.349703436 +0000 UTC m=+1319.470980962"
Nov 04 01:57:14.680553 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:14.680497    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:15.681643 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:15.681606    6116 patch_prober.go:29] interesting pod/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 04 01:57:15.682035 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:15.681666    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn" podUID="3d83b47f-120d-42b8-9e46-e7622dd29cfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 04 01:57:16.010748 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:16.010656    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4" podUID="79e5a497-975f-4f5f-b76f-20213e9f202d" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:57:16.112119 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:57:16.112081    6116 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-84cffd7d1248: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:57:16.112119 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:57:16.112112    6116 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:57:16.112341 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:57:16.112165    6116 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials podName:233cb7a1-2c92-4544-afe5-5aae953288e1 nodeName:}" failed. No retries permitted until 2026-11-04 01:59:18.112150689 +0000 UTC m=+1443.233428193 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials") pod "payload-processing-external-model-5497dccdcc-bbs4k" (UID: "233cb7a1-2c92-4544-afe5-5aae953288e1") : secret "agc-cred-84cffd7d1248" not found
Nov 04 01:57:16.680044 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:16.680008    6116 patch_prober.go:29] interesting pod/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 04 01:57:16.680227 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:16.680063    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn" podUID="3d83b47f-120d-42b8-9e46-e7622dd29cfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 04 01:57:17.680566 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:17.680531    6116 patch_prober.go:29] interesting pod/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 04 01:57:17.681003 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:17.680600    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn" podUID="3d83b47f-120d-42b8-9e46-e7622dd29cfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 04 01:57:17.829383 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:17.829354    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:57:18.680561 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:18.680526    6116 patch_prober.go:29] interesting pod/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 04 01:57:18.680752 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:18.680578    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn" podUID="3d83b47f-120d-42b8-9e46-e7622dd29cfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 04 01:57:19.234964 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:19.234929    6116 generic.go:356] "Generic (PLEG): container finished" podID="79e5a497-975f-4f5f-b76f-20213e9f202d" containerID="b0b59c99f0b7ed51bb08fabd420a651e252b300d0c3c3b6dcf73479ac8740a37" exitCode=0
Nov 04 01:57:19.235228 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:19.235210    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4" event={"ID":"79e5a497-975f-4f5f-b76f-20213e9f202d","Type":"ContainerDied","Data":"b0b59c99f0b7ed51bb08fabd420a651e252b300d0c3c3b6dcf73479ac8740a37"}
Nov 04 01:57:19.327290 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:19.327267    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4"
Nov 04 01:57:19.393806 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:19.393773    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79e5a497-975f-4f5f-b76f-20213e9f202d-kube-api-access-hjwjd\" (UniqueName: \"kubernetes.io/projected/79e5a497-975f-4f5f-b76f-20213e9f202d-kube-api-access-hjwjd\") pod \"79e5a497-975f-4f5f-b76f-20213e9f202d\" (UID: \"79e5a497-975f-4f5f-b76f-20213e9f202d\") "
Nov 04 01:57:19.394008 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:19.393896    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/79e5a497-975f-4f5f-b76f-20213e9f202d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79e5a497-975f-4f5f-b76f-20213e9f202d-plugins-config-volume\") pod \"79e5a497-975f-4f5f-b76f-20213e9f202d\" (UID: \"79e5a497-975f-4f5f-b76f-20213e9f202d\") "
Nov 04 01:57:19.394215 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:19.394192    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79e5a497-975f-4f5f-b76f-20213e9f202d-plugins-config-volume" pod "79e5a497-975f-4f5f-b76f-20213e9f202d" (UID: "79e5a497-975f-4f5f-b76f-20213e9f202d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:57:19.394537 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:19.394518    6116 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79e5a497-975f-4f5f-b76f-20213e9f202d-plugins-config-volume\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:19.396094 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:19.396073    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79e5a497-975f-4f5f-b76f-20213e9f202d-kube-api-access-hjwjd" pod "79e5a497-975f-4f5f-b76f-20213e9f202d" (UID: "79e5a497-975f-4f5f-b76f-20213e9f202d"). InnerVolumeSpecName "kube-api-access-hjwjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:19.496223 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:19.496188    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjwjd\" (UniqueName: \"kubernetes.io/projected/79e5a497-975f-4f5f-b76f-20213e9f202d-kube-api-access-hjwjd\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:19.679957 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:19.679916    6116 patch_prober.go:29] interesting pod/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 04 01:57:19.680143 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:19.679987    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn" podUID="3d83b47f-120d-42b8-9e46-e7622dd29cfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 04 01:57:20.069187 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:20.069099    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk"]
Nov 04 01:57:20.073928 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:20.073903    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-34d92457-tcskk"]
Nov 04 01:57:20.243051 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:20.243023    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4"
Nov 04 01:57:20.243220 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:20.243065    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4" event={"ID":"79e5a497-975f-4f5f-b76f-20213e9f202d","Type":"ContainerDied","Data":"1891fb8f522718ec14df1e4dbda366515dd2a7d3a73fc07f96030beff2fd2650"}
Nov 04 01:57:20.243220 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:20.243124    6116 scope.go:122] "RemoveContainer" containerID="b0b59c99f0b7ed51bb08fabd420a651e252b300d0c3c3b6dcf73479ac8740a37"
Nov 04 01:57:20.408635 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:20.408591    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4"]
Nov 04 01:57:20.413793 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:20.413758    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4"]
Nov 04 01:57:20.692533 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:20.688845    6116 patch_prober.go:29] interesting pod/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 04 01:57:20.692533 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:20.688904    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn" podUID="3d83b47f-120d-42b8-9e46-e7622dd29cfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 04 01:57:21.028666 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:21.028571    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="79e5a497-975f-4f5f-b76f-20213e9f202d" path="/var/lib/kubelet/pods/79e5a497-975f-4f5f-b76f-20213e9f202d/volumes"
Nov 04 01:57:21.029028 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:21.029011    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d949886a-cd12-48ed-9d9a-6fb9e6df4fb8" path="/var/lib/kubelet/pods/d949886a-cd12-48ed-9d9a-6fb9e6df4fb8/volumes"
Nov 04 01:57:21.913576 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:21.913543    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:21.914052 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:21.913895    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:21.916571 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:21.916546    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:24.208765 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:24.208734    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"]
Nov 04 01:57:24.271109 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:24.271055    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn" podUID="3d83b47f-120d-42b8-9e46-e7622dd29cfc" containerName="istio-proxy" containerID="cri-o://515727c60ae68944a2f959c648a3022295c14342034a50e62db0bd586939026f" gracePeriod=30
Nov 04 01:57:28.086074 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:28.086036    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"]
Nov 04 01:57:28.086550 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:28.086242    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9" podUID="93219663-8259-41a2-9b98-8488f493683d" containerName="istio-proxy" containerID="cri-o://7c68a1fa56695d720311617cf3072c1131f987c1cf0e1bf4ed24787aa602555c" gracePeriod=30
Nov 04 01:57:28.135870 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:28.135839    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"]
Nov 04 01:57:28.136066 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:28.136045    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67" podUID="1bd672da-f401-43ae-a6bd-52247191c3a9" containerName="istio-proxy" containerID="cri-o://e532191a456a0328a30d1287555aaf9e87340cbeffdc0c6170aba21af656f3d4" gracePeriod=30
Nov 04 01:57:29.534263 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.534240    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:29.639434 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.639344    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-workload-certs\") pod \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") "
Nov 04 01:57:29.639434 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.639399    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-podinfo\") pod \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") "
Nov 04 01:57:29.639434 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.639421    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-envoy\") pod \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") "
Nov 04 01:57:29.639731 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.639460    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istiod-ca-cert\") pod \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") "
Nov 04 01:57:29.639731 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.639496    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-token\" (UniqueName: \"kubernetes.io/projected/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-token\") pod \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") "
Nov 04 01:57:29.639731 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.639574    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-workload-socket\") pod \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") "
Nov 04 01:57:29.639731 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.639652    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-credential-socket\") pod \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") "
Nov 04 01:57:29.639731 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.639698    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-workload-certs" pod "3d83b47f-120d-42b8-9e46-e7622dd29cfc" (UID: "3d83b47f-120d-42b8-9e46-e7622dd29cfc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:29.639984 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.639733    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-data\") pod \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") "
Nov 04 01:57:29.639984 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.639776    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d83b47f-120d-42b8-9e46-e7622dd29cfc-kube-api-access-jzlrq\" (UniqueName: \"kubernetes.io/projected/3d83b47f-120d-42b8-9e46-e7622dd29cfc-kube-api-access-jzlrq\") pod \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\" (UID: \"3d83b47f-120d-42b8-9e46-e7622dd29cfc\") "
Nov 04 01:57:29.639984 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.639904    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istiod-ca-cert" pod "3d83b47f-120d-42b8-9e46-e7622dd29cfc" (UID: "3d83b47f-120d-42b8-9e46-e7622dd29cfc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:57:29.639984 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.639910    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-workload-socket" pod "3d83b47f-120d-42b8-9e46-e7622dd29cfc" (UID: "3d83b47f-120d-42b8-9e46-e7622dd29cfc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:29.640251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.640077    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-credential-socket" pod "3d83b47f-120d-42b8-9e46-e7622dd29cfc" (UID: "3d83b47f-120d-42b8-9e46-e7622dd29cfc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:29.640251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.640098    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-data" pod "3d83b47f-120d-42b8-9e46-e7622dd29cfc" (UID: "3d83b47f-120d-42b8-9e46-e7622dd29cfc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:29.640356 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.640304    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:29.640356 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.640323    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:29.640356 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.640338    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:29.640356 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.640354    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:29.640740 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.640368    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:29.642275 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.642246    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-envoy" pod "3d83b47f-120d-42b8-9e46-e7622dd29cfc" (UID: "3d83b47f-120d-42b8-9e46-e7622dd29cfc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:29.642390 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.642252    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-podinfo" pod "3d83b47f-120d-42b8-9e46-e7622dd29cfc" (UID: "3d83b47f-120d-42b8-9e46-e7622dd29cfc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:57:29.642854 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.642829    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-token" pod "3d83b47f-120d-42b8-9e46-e7622dd29cfc" (UID: "3d83b47f-120d-42b8-9e46-e7622dd29cfc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:29.642935 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.642861    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d83b47f-120d-42b8-9e46-e7622dd29cfc-kube-api-access-jzlrq" pod "3d83b47f-120d-42b8-9e46-e7622dd29cfc" (UID: "3d83b47f-120d-42b8-9e46-e7622dd29cfc"). InnerVolumeSpecName "kube-api-access-jzlrq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:29.741333 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.741292    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:29.741333 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.741325    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:29.741333 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.741334    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d83b47f-120d-42b8-9e46-e7622dd29cfc-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:29.741333 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:29.741342    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzlrq\" (UniqueName: \"kubernetes.io/projected/3d83b47f-120d-42b8-9e46-e7622dd29cfc-kube-api-access-jzlrq\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:30.310893 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:30.310860    6116 generic.go:356] "Generic (PLEG): container finished" podID="3d83b47f-120d-42b8-9e46-e7622dd29cfc" containerID="515727c60ae68944a2f959c648a3022295c14342034a50e62db0bd586939026f" exitCode=0
Nov 04 01:57:30.311069 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:30.310910    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"
Nov 04 01:57:30.311069 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:30.310945    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn" event={"ID":"3d83b47f-120d-42b8-9e46-e7622dd29cfc","Type":"ContainerDied","Data":"515727c60ae68944a2f959c648a3022295c14342034a50e62db0bd586939026f"}
Nov 04 01:57:30.311069 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:30.311002    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn" event={"ID":"3d83b47f-120d-42b8-9e46-e7622dd29cfc","Type":"ContainerDied","Data":"a29bcc1c36321f9105005f7b7201cc26a467a617af560e0c972dee1df9540016"}
Nov 04 01:57:30.311069 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:30.311031    6116 scope.go:122] "RemoveContainer" containerID="515727c60ae68944a2f959c648a3022295c14342034a50e62db0bd586939026f"
Nov 04 01:57:30.321426 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:30.321406    6116 scope.go:122] "RemoveContainer" containerID="515727c60ae68944a2f959c648a3022295c14342034a50e62db0bd586939026f"
Nov 04 01:57:30.321711 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:57:30.321693    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"515727c60ae68944a2f959c648a3022295c14342034a50e62db0bd586939026f\": container with ID starting with 515727c60ae68944a2f959c648a3022295c14342034a50e62db0bd586939026f not found: ID does not exist" containerID="515727c60ae68944a2f959c648a3022295c14342034a50e62db0bd586939026f"
Nov 04 01:57:30.321771 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:30.321720    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"515727c60ae68944a2f959c648a3022295c14342034a50e62db0bd586939026f"} err="failed to get container status \"515727c60ae68944a2f959c648a3022295c14342034a50e62db0bd586939026f\": rpc error: code = NotFound desc = could not find container \"515727c60ae68944a2f959c648a3022295c14342034a50e62db0bd586939026f\": container with ID starting with 515727c60ae68944a2f959c648a3022295c14342034a50e62db0bd586939026f not found: ID does not exist"
Nov 04 01:57:30.475577 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:30.475543    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"]
Nov 04 01:57:30.526325 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:30.526297    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn"]
Nov 04 01:57:30.910002 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:30.909966    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"]
Nov 04 01:57:30.910412 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:30.910175    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh" podUID="89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" containerName="istio-proxy" containerID="cri-o://36de27cba574e95cee6d78ef560b41db4885372d5be411ca1a530ec057a52193" gracePeriod=30
Nov 04 01:57:31.027836 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:31.027800    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d83b47f-120d-42b8-9e46-e7622dd29cfc" path="/var/lib/kubelet/pods/3d83b47f-120d-42b8-9e46-e7622dd29cfc/volumes"
Nov 04 01:57:32.820421 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:32.820383    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:57:32.820841 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:32.820440    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67" podUID="1bd672da-f401-43ae-a6bd-52247191c3a9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:57:33.332035 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.332004    6116 generic.go:356] "Generic (PLEG): container finished" podID="1bd672da-f401-43ae-a6bd-52247191c3a9" containerID="e532191a456a0328a30d1287555aaf9e87340cbeffdc0c6170aba21af656f3d4" exitCode=0
Nov 04 01:57:33.332231 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.332071    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67" event={"ID":"1bd672da-f401-43ae-a6bd-52247191c3a9","Type":"ContainerDied","Data":"e532191a456a0328a30d1287555aaf9e87340cbeffdc0c6170aba21af656f3d4"}
Nov 04 01:57:33.334224 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.334195    6116 generic.go:356] "Generic (PLEG): container finished" podID="93219663-8259-41a2-9b98-8488f493683d" containerID="7c68a1fa56695d720311617cf3072c1131f987c1cf0e1bf4ed24787aa602555c" exitCode=0
Nov 04 01:57:33.334327 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.334262    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9" event={"ID":"93219663-8259-41a2-9b98-8488f493683d","Type":"ContainerDied","Data":"7c68a1fa56695d720311617cf3072c1131f987c1cf0e1bf4ed24787aa602555c"}
Nov 04 01:57:33.393051 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.393030    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:57:33.422331 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.422308    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:57:33.482620 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.482585    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-workload-socket\") pod \"1bd672da-f401-43ae-a6bd-52247191c3a9\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") "
Nov 04 01:57:33.482778 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.482637    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-token\" (UniqueName: \"kubernetes.io/projected/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-token\") pod \"1bd672da-f401-43ae-a6bd-52247191c3a9\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") "
Nov 04 01:57:33.482778 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.482696    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-podinfo\") pod \"1bd672da-f401-43ae-a6bd-52247191c3a9\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") "
Nov 04 01:57:33.482778 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.482752    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93219663-8259-41a2-9b98-8488f493683d-kube-api-access-zg5ff\" (UniqueName: \"kubernetes.io/projected/93219663-8259-41a2-9b98-8488f493683d-kube-api-access-zg5ff\") pod \"93219663-8259-41a2-9b98-8488f493683d\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") "
Nov 04 01:57:33.482944 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.482797    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-data\") pod \"1bd672da-f401-43ae-a6bd-52247191c3a9\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") "
Nov 04 01:57:33.482944 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.482822    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1bd672da-f401-43ae-a6bd-52247191c3a9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bd672da-f401-43ae-a6bd-52247191c3a9-istiod-ca-cert\") pod \"1bd672da-f401-43ae-a6bd-52247191c3a9\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") "
Nov 04 01:57:33.482944 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.482892    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-workload-certs\") pod \"1bd672da-f401-43ae-a6bd-52247191c3a9\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") "
Nov 04 01:57:33.482944 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.482918    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-envoy\") pod \"1bd672da-f401-43ae-a6bd-52247191c3a9\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") "
Nov 04 01:57:33.482944 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.482915    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-workload-socket" pod "1bd672da-f401-43ae-a6bd-52247191c3a9" (UID: "1bd672da-f401-43ae-a6bd-52247191c3a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:33.483190 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.482976    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-istio-envoy\") pod \"93219663-8259-41a2-9b98-8488f493683d\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") "
Nov 04 01:57:33.483190 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.483045    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-credential-socket\") pod \"1bd672da-f401-43ae-a6bd-52247191c3a9\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") "
Nov 04 01:57:33.483190 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.483076    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-istio-data\") pod \"93219663-8259-41a2-9b98-8488f493683d\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") "
Nov 04 01:57:33.483190 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.483111    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-data" pod "1bd672da-f401-43ae-a6bd-52247191c3a9" (UID: "1bd672da-f401-43ae-a6bd-52247191c3a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:33.483190 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.483127    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-workload-socket\") pod \"93219663-8259-41a2-9b98-8488f493683d\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") "
Nov 04 01:57:33.483190 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.483145    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-workload-certs" pod "1bd672da-f401-43ae-a6bd-52247191c3a9" (UID: "1bd672da-f401-43ae-a6bd-52247191c3a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:33.483190 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.483159    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1bd672da-f401-43ae-a6bd-52247191c3a9-kube-api-access-5ktmz\" (UniqueName: \"kubernetes.io/projected/1bd672da-f401-43ae-a6bd-52247191c3a9-kube-api-access-5ktmz\") pod \"1bd672da-f401-43ae-a6bd-52247191c3a9\" (UID: \"1bd672da-f401-43ae-a6bd-52247191c3a9\") "
Nov 04 01:57:33.483190 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.483185    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/93219663-8259-41a2-9b98-8488f493683d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93219663-8259-41a2-9b98-8488f493683d-istio-podinfo\") pod \"93219663-8259-41a2-9b98-8488f493683d\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") "
Nov 04 01:57:33.483626 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.483236    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-workload-certs\") pod \"93219663-8259-41a2-9b98-8488f493683d\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") "
Nov 04 01:57:33.483626 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.483313    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93219663-8259-41a2-9b98-8488f493683d-istio-token\" (UniqueName: \"kubernetes.io/projected/93219663-8259-41a2-9b98-8488f493683d-istio-token\") pod \"93219663-8259-41a2-9b98-8488f493683d\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") "
Nov 04 01:57:33.483626 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.483390    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-credential-socket\") pod \"93219663-8259-41a2-9b98-8488f493683d\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") "
Nov 04 01:57:33.483626 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.483415    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93219663-8259-41a2-9b98-8488f493683d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93219663-8259-41a2-9b98-8488f493683d-istiod-ca-cert\") pod \"93219663-8259-41a2-9b98-8488f493683d\" (UID: \"93219663-8259-41a2-9b98-8488f493683d\") "
Nov 04 01:57:33.483826 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.483310    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-credential-socket" pod "1bd672da-f401-43ae-a6bd-52247191c3a9" (UID: "1bd672da-f401-43ae-a6bd-52247191c3a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:33.484743 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.484103    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-istio-data" pod "93219663-8259-41a2-9b98-8488f493683d" (UID: "93219663-8259-41a2-9b98-8488f493683d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:33.484743 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.483510    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bd672da-f401-43ae-a6bd-52247191c3a9-istiod-ca-cert" pod "1bd672da-f401-43ae-a6bd-52247191c3a9" (UID: "1bd672da-f401-43ae-a6bd-52247191c3a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:57:33.484743 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.484106    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:33.484743 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.484160    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:33.484743 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.484177    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:33.484743 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.484191    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:33.484743 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.484371    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-workload-socket" pod "93219663-8259-41a2-9b98-8488f493683d" (UID: "93219663-8259-41a2-9b98-8488f493683d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:33.484743 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.484484    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-workload-certs" pod "93219663-8259-41a2-9b98-8488f493683d" (UID: "93219663-8259-41a2-9b98-8488f493683d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:33.484743 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.484576    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-credential-socket" pod "93219663-8259-41a2-9b98-8488f493683d" (UID: "93219663-8259-41a2-9b98-8488f493683d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:33.484743 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.484718    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93219663-8259-41a2-9b98-8488f493683d-istiod-ca-cert" pod "93219663-8259-41a2-9b98-8488f493683d" (UID: "93219663-8259-41a2-9b98-8488f493683d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:57:33.485898 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.485859    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-token" pod "1bd672da-f401-43ae-a6bd-52247191c3a9" (UID: "1bd672da-f401-43ae-a6bd-52247191c3a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:33.486230 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.486196    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-istio-envoy" pod "93219663-8259-41a2-9b98-8488f493683d" (UID: "93219663-8259-41a2-9b98-8488f493683d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:33.486326 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.486261    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-envoy" pod "1bd672da-f401-43ae-a6bd-52247191c3a9" (UID: "1bd672da-f401-43ae-a6bd-52247191c3a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:33.486326 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.486271    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93219663-8259-41a2-9b98-8488f493683d-istio-token" pod "93219663-8259-41a2-9b98-8488f493683d" (UID: "93219663-8259-41a2-9b98-8488f493683d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:33.486326 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.486282    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-podinfo" pod "1bd672da-f401-43ae-a6bd-52247191c3a9" (UID: "1bd672da-f401-43ae-a6bd-52247191c3a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:57:33.486485 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.486378    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bd672da-f401-43ae-a6bd-52247191c3a9-kube-api-access-5ktmz" pod "1bd672da-f401-43ae-a6bd-52247191c3a9" (UID: "1bd672da-f401-43ae-a6bd-52247191c3a9"). InnerVolumeSpecName "kube-api-access-5ktmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:33.486527 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.486496    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93219663-8259-41a2-9b98-8488f493683d-kube-api-access-zg5ff" pod "93219663-8259-41a2-9b98-8488f493683d" (UID: "93219663-8259-41a2-9b98-8488f493683d"). InnerVolumeSpecName "kube-api-access-zg5ff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:33.486984 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.486969    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93219663-8259-41a2-9b98-8488f493683d-istio-podinfo" pod "93219663-8259-41a2-9b98-8488f493683d" (UID: "93219663-8259-41a2-9b98-8488f493683d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:57:33.585600 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.585563    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:33.585600 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.585594    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:33.585811 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.585610    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:33.585811 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.585624    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:33.585811 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.585633    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ktmz\" (UniqueName: \"kubernetes.io/projected/1bd672da-f401-43ae-a6bd-52247191c3a9-kube-api-access-5ktmz\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:33.585811 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.585642    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93219663-8259-41a2-9b98-8488f493683d-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:33.585811 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.585651    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:33.585811 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.585659    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93219663-8259-41a2-9b98-8488f493683d-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:33.585811 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.585666    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93219663-8259-41a2-9b98-8488f493683d-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:33.585811 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.585680    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93219663-8259-41a2-9b98-8488f493683d-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:33.585811 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.585692    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:33.585811 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.585705    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bd672da-f401-43ae-a6bd-52247191c3a9-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:33.585811 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.585717    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zg5ff\" (UniqueName: \"kubernetes.io/projected/93219663-8259-41a2-9b98-8488f493683d-kube-api-access-zg5ff\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:33.585811 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:33.585729    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bd672da-f401-43ae-a6bd-52247191c3a9-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:34.342231 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:34.342196    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67" event={"ID":"1bd672da-f401-43ae-a6bd-52247191c3a9","Type":"ContainerDied","Data":"33f78a88170cedf84a5ba2b4a3b5c9ef0188aa57910f96e16961de259787f6e5"}
Nov 04 01:57:34.342725 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:34.342272    6116 scope.go:122] "RemoveContainer" containerID="e532191a456a0328a30d1287555aaf9e87340cbeffdc0c6170aba21af656f3d4"
Nov 04 01:57:34.342725 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:34.342208    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"
Nov 04 01:57:34.343912 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:34.343887    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9" event={"ID":"93219663-8259-41a2-9b98-8488f493683d","Type":"ContainerDied","Data":"c2905fc40480fafa18f6364fc556e173b93db5f266e3863a1aebd3f3c1c99170"}
Nov 04 01:57:34.344010 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:34.343901    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"
Nov 04 01:57:34.356556 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:34.356532    6116 scope.go:122] "RemoveContainer" containerID="7c68a1fa56695d720311617cf3072c1131f987c1cf0e1bf4ed24787aa602555c"
Nov 04 01:57:34.557432 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:34.557397    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"]
Nov 04 01:57:34.561618 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:34.561592    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67"]
Nov 04 01:57:34.681915 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:34.681878    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"]
Nov 04 01:57:34.692380 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:34.692335    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9"]
Nov 04 01:57:35.029352 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:35.029271    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1bd672da-f401-43ae-a6bd-52247191c3a9" path="/var/lib/kubelet/pods/1bd672da-f401-43ae-a6bd-52247191c3a9/volumes"
Nov 04 01:57:35.029756 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:35.029741    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93219663-8259-41a2-9b98-8488f493683d" path="/var/lib/kubelet/pods/93219663-8259-41a2-9b98-8488f493683d/volumes"
Nov 04 01:57:36.158773 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.158743    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:57:36.213985 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.213900    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-data\") pod \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") "
Nov 04 01:57:36.213985 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.213965    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-credential-socket\") pod \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") "
Nov 04 01:57:36.214208 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.214013    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-workload-socket\") pod \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") "
Nov 04 01:57:36.214208 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.214039    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-podinfo\") pod \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") "
Nov 04 01:57:36.214208 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.214114    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-token\" (UniqueName: \"kubernetes.io/projected/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-token\") pod \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") "
Nov 04 01:57:36.214208 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.214171    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-data" pod "89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" (UID: "89a13dcf-c8a3-4092-8ff0-d63fb3ae874f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:36.214208 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.214187    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istiod-ca-cert\") pod \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") "
Nov 04 01:57:36.214495 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.214223    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-credential-socket" pod "89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" (UID: "89a13dcf-c8a3-4092-8ff0-d63fb3ae874f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:36.214495 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.214242    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-envoy\") pod \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") "
Nov 04 01:57:36.214495 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.214275    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-workload-socket" pod "89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" (UID: "89a13dcf-c8a3-4092-8ff0-d63fb3ae874f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:36.214495 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.214322    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-kube-api-access-2zpd7\" (UniqueName: \"kubernetes.io/projected/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-kube-api-access-2zpd7\") pod \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") "
Nov 04 01:57:36.214495 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.214375    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-workload-certs\") pod \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\" (UID: \"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f\") "
Nov 04 01:57:36.214763 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.214672    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-workload-certs" pod "89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" (UID: "89a13dcf-c8a3-4092-8ff0-d63fb3ae874f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:36.214763 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.214696    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istiod-ca-cert" pod "89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" (UID: "89a13dcf-c8a3-4092-8ff0-d63fb3ae874f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:57:36.214986 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.214889    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:36.214986 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.214906    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:36.214986 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.214921    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:36.214986 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.214934    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:36.214986 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.214942    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:36.216515 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.216491    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-envoy" pod "89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" (UID: "89a13dcf-c8a3-4092-8ff0-d63fb3ae874f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:57:36.216635 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.216568    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-kube-api-access-2zpd7" pod "89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" (UID: "89a13dcf-c8a3-4092-8ff0-d63fb3ae874f"). InnerVolumeSpecName "kube-api-access-2zpd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:36.216869 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.216855    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-token" pod "89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" (UID: "89a13dcf-c8a3-4092-8ff0-d63fb3ae874f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:57:36.216925 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.216911    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-podinfo" pod "89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" (UID: "89a13dcf-c8a3-4092-8ff0-d63fb3ae874f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:57:36.315817 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.315771    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:36.315817 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.315811    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:36.315817 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.315822    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:36.315817 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.315832    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zpd7\" (UniqueName: \"kubernetes.io/projected/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f-kube-api-access-2zpd7\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:57:36.361398 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.361362    6116 generic.go:356] "Generic (PLEG): container finished" podID="89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" containerID="36de27cba574e95cee6d78ef560b41db4885372d5be411ca1a530ec057a52193" exitCode=0
Nov 04 01:57:36.361605 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.361412    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"
Nov 04 01:57:36.361605 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.361441    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh" event={"ID":"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f","Type":"ContainerDied","Data":"36de27cba574e95cee6d78ef560b41db4885372d5be411ca1a530ec057a52193"}
Nov 04 01:57:36.361605 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.361527    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh" event={"ID":"89a13dcf-c8a3-4092-8ff0-d63fb3ae874f","Type":"ContainerDied","Data":"86818ead554be7d7346e46b13c280c9e0a060169a07ff5237b11612730745d82"}
Nov 04 01:57:36.361605 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.361553    6116 scope.go:122] "RemoveContainer" containerID="36de27cba574e95cee6d78ef560b41db4885372d5be411ca1a530ec057a52193"
Nov 04 01:57:36.372614 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.372593    6116 scope.go:122] "RemoveContainer" containerID="36de27cba574e95cee6d78ef560b41db4885372d5be411ca1a530ec057a52193"
Nov 04 01:57:36.372886 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:57:36.372867    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36de27cba574e95cee6d78ef560b41db4885372d5be411ca1a530ec057a52193\": container with ID starting with 36de27cba574e95cee6d78ef560b41db4885372d5be411ca1a530ec057a52193 not found: ID does not exist" containerID="36de27cba574e95cee6d78ef560b41db4885372d5be411ca1a530ec057a52193"
Nov 04 01:57:36.372952 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.372901    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36de27cba574e95cee6d78ef560b41db4885372d5be411ca1a530ec057a52193"} err="failed to get container status \"36de27cba574e95cee6d78ef560b41db4885372d5be411ca1a530ec057a52193\": rpc error: code = NotFound desc = could not find container \"36de27cba574e95cee6d78ef560b41db4885372d5be411ca1a530ec057a52193\": container with ID starting with 36de27cba574e95cee6d78ef560b41db4885372d5be411ca1a530ec057a52193 not found: ID does not exist"
Nov 04 01:57:36.527175 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.527097    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"]
Nov 04 01:57:36.532505 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:36.532480    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh"]
Nov 04 01:57:37.027646 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:37.027616    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" path="/var/lib/kubelet/pods/89a13dcf-c8a3-4092-8ff0-d63fb3ae874f/volumes"
Nov 04 01:57:37.052396 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:37.052365    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m"]
Nov 04 01:57:37.054795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:37.054775    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-497ccb8c-nr92m"]
Nov 04 01:57:39.027931 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:39.027901    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56" path="/var/lib/kubelet/pods/e04dcc2f-f680-4bd3-9cf4-5b7a00f3cc56/volumes"
Nov 04 01:57:41.991623 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:57:41.991588    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:57:48.094881 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:48.094847    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:57:54.048314 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:54.048279    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774"]
Nov 04 01:57:54.058508 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:54.058483    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d3605df-hx774"]
Nov 04 01:57:55.028372 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:57:55.028334    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a334c719-70fa-4886-bd0f-57d102af0a77" path="/var/lib/kubelet/pods/a334c719-70fa-4886-bd0f-57d102af0a77/volumes"
Nov 04 01:58:04.741399 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:04.741362    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"]
Nov 04 01:58:04.741859 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:04.741593    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6" podUID="a649747c-ccad-4629-8be4-43fc827eb3f9" containerName="istio-proxy" containerID="cri-o://7810a6f5c8c7c6c57582c3cc5da64104795d7970d7444b97fcee6c686d30c92c" gracePeriod=30
Nov 04 01:58:06.339970 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.339931    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm"]
Nov 04 01:58:06.340516 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.340491    6116 cpu_manager.go:414] "containerMap: removing container" podUID="1bd672da-f401-43ae-a6bd-52247191c3a9" containerName="istio-proxy"
Nov 04 01:58:06.340516 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.340507    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1bd672da-f401-43ae-a6bd-52247191c3a9" containerName="istio-proxy"
Nov 04 01:58:06.340630 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.340537    6116 cpu_manager.go:414] "containerMap: removing container" podUID="79e5a497-975f-4f5f-b76f-20213e9f202d" containerName="payload-pre-processing"
Nov 04 01:58:06.340630 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.340543    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79e5a497-975f-4f5f-b76f-20213e9f202d" containerName="payload-pre-processing"
Nov 04 01:58:06.340630 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.340552    6116 cpu_manager.go:414] "containerMap: removing container" podUID="93219663-8259-41a2-9b98-8488f493683d" containerName="istio-proxy"
Nov 04 01:58:06.340630 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.340556    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93219663-8259-41a2-9b98-8488f493683d" containerName="istio-proxy"
Nov 04 01:58:06.340630 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.340564    6116 cpu_manager.go:414] "containerMap: removing container" podUID="3d83b47f-120d-42b8-9e46-e7622dd29cfc" containerName="istio-proxy"
Nov 04 01:58:06.340630 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.340568    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d83b47f-120d-42b8-9e46-e7622dd29cfc" containerName="istio-proxy"
Nov 04 01:58:06.340630 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.340574    6116 cpu_manager.go:414] "containerMap: removing container" podUID="89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" containerName="istio-proxy"
Nov 04 01:58:06.340630 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.340578    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" containerName="istio-proxy"
Nov 04 01:58:06.340855 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.340661    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="79e5a497-975f-4f5f-b76f-20213e9f202d" containerName="payload-pre-processing"
Nov 04 01:58:06.340855 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.340671    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="93219663-8259-41a2-9b98-8488f493683d" containerName="istio-proxy"
Nov 04 01:58:06.340855 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.340681    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d83b47f-120d-42b8-9e46-e7622dd29cfc" containerName="istio-proxy"
Nov 04 01:58:06.340855 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.340689    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="89a13dcf-c8a3-4092-8ff0-d63fb3ae874f" containerName="istio-proxy"
Nov 04 01:58:06.340855 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.340698    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="1bd672da-f401-43ae-a6bd-52247191c3a9" containerName="istio-proxy"
Nov 04 01:58:06.343909 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.343892    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm"
Nov 04 01:58:06.397541 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.397503    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 01:58:06.397723 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.397540    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:58:06.447105 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.447068    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:58:06.447307 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.447112    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bj8b6\""
Nov 04 01:58:06.449821 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.449802    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm"]
Nov 04 01:58:06.456654 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.456628    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-65edf5d0-f7bhm\" (UID: \"6ddcfdc7-9d1d-4698-b43d-6a673d31497a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm"
Nov 04 01:58:06.456768 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.456664    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zddb\" (UniqueName: \"kubernetes.io/projected/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-kube-api-access-8zddb\") pod \"maas-api-revoke-sub-65edf5d0-f7bhm\" (UID: \"6ddcfdc7-9d1d-4698-b43d-6a673d31497a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm"
Nov 04 01:58:06.456768 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.456701    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-65edf5d0-f7bhm\" (UID: \"6ddcfdc7-9d1d-4698-b43d-6a673d31497a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm"
Nov 04 01:58:06.558496 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.558436    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-65edf5d0-f7bhm\" (UID: \"6ddcfdc7-9d1d-4698-b43d-6a673d31497a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm"
Nov 04 01:58:06.558681 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.558515    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zddb\" (UniqueName: \"kubernetes.io/projected/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-kube-api-access-8zddb\") pod \"maas-api-revoke-sub-65edf5d0-f7bhm\" (UID: \"6ddcfdc7-9d1d-4698-b43d-6a673d31497a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm"
Nov 04 01:58:06.558681 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.558555    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-65edf5d0-f7bhm\" (UID: \"6ddcfdc7-9d1d-4698-b43d-6a673d31497a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm"
Nov 04 01:58:06.559164 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.559144    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-65edf5d0-f7bhm\" (UID: \"6ddcfdc7-9d1d-4698-b43d-6a673d31497a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm"
Nov 04 01:58:06.560922 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.560904    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-65edf5d0-f7bhm\" (UID: \"6ddcfdc7-9d1d-4698-b43d-6a673d31497a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm"
Nov 04 01:58:06.619541 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.619430    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zddb\" (UniqueName: \"kubernetes.io/projected/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-kube-api-access-8zddb\") pod \"maas-api-revoke-sub-65edf5d0-f7bhm\" (UID: \"6ddcfdc7-9d1d-4698-b43d-6a673d31497a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm"
Nov 04 01:58:06.657470 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:06.657404    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm"
Nov 04 01:58:07.036795 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:07.036768    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm"]
Nov 04 01:58:07.037760 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:58:07.037658    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ddcfdc7_9d1d_4698_b43d_6a673d31497a.slice/crio-25f2a6a50668ca6ef956fa94a0262baaf49c791dae07cde5f8cd696a36cb3b03 WatchSource:0}: Error finding container 25f2a6a50668ca6ef956fa94a0262baaf49c791dae07cde5f8cd696a36cb3b03: Status 404 returned error can't find the container with id 25f2a6a50668ca6ef956fa94a0262baaf49c791dae07cde5f8cd696a36cb3b03
Nov 04 01:58:07.560754 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:07.560709    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm" event={"ID":"6ddcfdc7-9d1d-4698-b43d-6a673d31497a","Type":"ContainerStarted","Data":"a58c960c125ba1ac09d2d5118166c4ac081c82896d0710db17591ff59c1fbf14"}
Nov 04 01:58:07.561152 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:07.560772    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm" event={"ID":"6ddcfdc7-9d1d-4698-b43d-6a673d31497a","Type":"ContainerStarted","Data":"25f2a6a50668ca6ef956fa94a0262baaf49c791dae07cde5f8cd696a36cb3b03"}
Nov 04 01:58:07.677377 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:07.677319    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm" podStartSLOduration=1.677304073 podStartE2EDuration="1.677304073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:58:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:58:07.675105889 +0000 UTC m=+1372.796383414" watchObservedRunningTime="2026-11-04 01:58:07.677304073 +0000 UTC m=+1372.798581623"
Nov 04 01:58:08.568370 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:08.568330    6116 generic.go:356] "Generic (PLEG): container finished" podID="6ddcfdc7-9d1d-4698-b43d-6a673d31497a" containerID="a58c960c125ba1ac09d2d5118166c4ac081c82896d0710db17591ff59c1fbf14" exitCode=0
Nov 04 01:58:08.568837 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:08.568416    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm" event={"ID":"6ddcfdc7-9d1d-4698-b43d-6a673d31497a","Type":"ContainerDied","Data":"a58c960c125ba1ac09d2d5118166c4ac081c82896d0710db17591ff59c1fbf14"}
Nov 04 01:58:09.183244 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:09.183208    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:58:09.714357 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:09.714331    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm"
Nov 04 01:58:09.798718 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:09.798689    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-maas-api-xks-ca\") pod \"6ddcfdc7-9d1d-4698-b43d-6a673d31497a\" (UID: \"6ddcfdc7-9d1d-4698-b43d-6a673d31497a\") "
Nov 04 01:58:09.798866 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:09.798750    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-kube-api-access-8zddb\" (UniqueName: \"kubernetes.io/projected/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-kube-api-access-8zddb\") pod \"6ddcfdc7-9d1d-4698-b43d-6a673d31497a\" (UID: \"6ddcfdc7-9d1d-4698-b43d-6a673d31497a\") "
Nov 04 01:58:09.798909 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:09.798876    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-maas-api-openshift-ca\") pod \"6ddcfdc7-9d1d-4698-b43d-6a673d31497a\" (UID: \"6ddcfdc7-9d1d-4698-b43d-6a673d31497a\") "
Nov 04 01:58:09.799216 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:09.799193    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-maas-api-openshift-ca" pod "6ddcfdc7-9d1d-4698-b43d-6a673d31497a" (UID: "6ddcfdc7-9d1d-4698-b43d-6a673d31497a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:58:09.800912 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:09.800890    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-maas-api-xks-ca" pod "6ddcfdc7-9d1d-4698-b43d-6a673d31497a" (UID: "6ddcfdc7-9d1d-4698-b43d-6a673d31497a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:58:09.801003 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:09.800983    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-kube-api-access-8zddb" pod "6ddcfdc7-9d1d-4698-b43d-6a673d31497a" (UID: "6ddcfdc7-9d1d-4698-b43d-6a673d31497a"). InnerVolumeSpecName "kube-api-access-8zddb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:09.900264 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:09.900231    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zddb\" (UniqueName: \"kubernetes.io/projected/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-kube-api-access-8zddb\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:09.900264 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:09.900261    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-maas-api-openshift-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:09.900264 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:09.900270    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6ddcfdc7-9d1d-4698-b43d-6a673d31497a-maas-api-xks-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:09.999072 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:09.999044    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:58:10.102488 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.102371    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-credential-socket\") pod \"a649747c-ccad-4629-8be4-43fc827eb3f9\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") "
Nov 04 01:58:10.102488 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.102432    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-envoy\") pod \"a649747c-ccad-4629-8be4-43fc827eb3f9\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") "
Nov 04 01:58:10.102730 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.102549    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-token\" (UniqueName: \"kubernetes.io/projected/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-token\") pod \"a649747c-ccad-4629-8be4-43fc827eb3f9\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") "
Nov 04 01:58:10.102730 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.102575    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-data\") pod \"a649747c-ccad-4629-8be4-43fc827eb3f9\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") "
Nov 04 01:58:10.102730 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.102602    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a649747c-ccad-4629-8be4-43fc827eb3f9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a649747c-ccad-4629-8be4-43fc827eb3f9-istiod-ca-cert\") pod \"a649747c-ccad-4629-8be4-43fc827eb3f9\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") "
Nov 04 01:58:10.102730 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.102674    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-workload-certs\") pod \"a649747c-ccad-4629-8be4-43fc827eb3f9\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") "
Nov 04 01:58:10.102730 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.102706    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-workload-socket\") pod \"a649747c-ccad-4629-8be4-43fc827eb3f9\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") "
Nov 04 01:58:10.102978 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.102771    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-podinfo\") pod \"a649747c-ccad-4629-8be4-43fc827eb3f9\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") "
Nov 04 01:58:10.102978 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.102774    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-credential-socket" pod "a649747c-ccad-4629-8be4-43fc827eb3f9" (UID: "a649747c-ccad-4629-8be4-43fc827eb3f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:10.102978 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.102827    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a649747c-ccad-4629-8be4-43fc827eb3f9-kube-api-access-z5npq\" (UniqueName: \"kubernetes.io/projected/a649747c-ccad-4629-8be4-43fc827eb3f9-kube-api-access-z5npq\") pod \"a649747c-ccad-4629-8be4-43fc827eb3f9\" (UID: \"a649747c-ccad-4629-8be4-43fc827eb3f9\") "
Nov 04 01:58:10.103126 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.102994    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-workload-certs" pod "a649747c-ccad-4629-8be4-43fc827eb3f9" (UID: "a649747c-ccad-4629-8be4-43fc827eb3f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:10.103126 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.103050    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-workload-socket" pod "a649747c-ccad-4629-8be4-43fc827eb3f9" (UID: "a649747c-ccad-4629-8be4-43fc827eb3f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:10.103126 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.103105    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a649747c-ccad-4629-8be4-43fc827eb3f9-istiod-ca-cert" pod "a649747c-ccad-4629-8be4-43fc827eb3f9" (UID: "a649747c-ccad-4629-8be4-43fc827eb3f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:58:10.103296 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.103275    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-data" pod "a649747c-ccad-4629-8be4-43fc827eb3f9" (UID: "a649747c-ccad-4629-8be4-43fc827eb3f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:10.103549 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.103461    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:10.103549 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.103483    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a649747c-ccad-4629-8be4-43fc827eb3f9-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:10.103549 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.103496    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:10.103549 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.103509    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:10.103549 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.103526    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:10.104923 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.104896    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-envoy" pod "a649747c-ccad-4629-8be4-43fc827eb3f9" (UID: "a649747c-ccad-4629-8be4-43fc827eb3f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:10.105265 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.105246    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-token" pod "a649747c-ccad-4629-8be4-43fc827eb3f9" (UID: "a649747c-ccad-4629-8be4-43fc827eb3f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:10.105541 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.105513    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-podinfo" pod "a649747c-ccad-4629-8be4-43fc827eb3f9" (UID: "a649747c-ccad-4629-8be4-43fc827eb3f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:58:10.105541 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.105532    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a649747c-ccad-4629-8be4-43fc827eb3f9-kube-api-access-z5npq" pod "a649747c-ccad-4629-8be4-43fc827eb3f9" (UID: "a649747c-ccad-4629-8be4-43fc827eb3f9"). InnerVolumeSpecName "kube-api-access-z5npq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:10.204715 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.204672    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:10.204715 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.204712    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5npq\" (UniqueName: \"kubernetes.io/projected/a649747c-ccad-4629-8be4-43fc827eb3f9-kube-api-access-z5npq\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:10.204925 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.204728    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:10.204925 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.204745    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a649747c-ccad-4629-8be4-43fc827eb3f9-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:10.206838 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.206815    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"]
Nov 04 01:58:10.226421 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.226389    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"]
Nov 04 01:58:10.584961 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.584921    6116 generic.go:356] "Generic (PLEG): container finished" podID="a649747c-ccad-4629-8be4-43fc827eb3f9" containerID="7810a6f5c8c7c6c57582c3cc5da64104795d7970d7444b97fcee6c686d30c92c" exitCode=0
Nov 04 01:58:10.585185 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.584972    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"
Nov 04 01:58:10.585185 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.585004    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6" event={"ID":"a649747c-ccad-4629-8be4-43fc827eb3f9","Type":"ContainerDied","Data":"7810a6f5c8c7c6c57582c3cc5da64104795d7970d7444b97fcee6c686d30c92c"}
Nov 04 01:58:10.585185 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.585069    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6" event={"ID":"a649747c-ccad-4629-8be4-43fc827eb3f9","Type":"ContainerDied","Data":"d99c56bc2ce5904193baae14afc6631a4d7723627c771202c8c299541baa5d9b"}
Nov 04 01:58:10.585185 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.585097    6116 scope.go:122] "RemoveContainer" containerID="7810a6f5c8c7c6c57582c3cc5da64104795d7970d7444b97fcee6c686d30c92c"
Nov 04 01:58:10.587090 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.586951    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm" event={"ID":"6ddcfdc7-9d1d-4698-b43d-6a673d31497a","Type":"ContainerDied","Data":"25f2a6a50668ca6ef956fa94a0262baaf49c791dae07cde5f8cd696a36cb3b03"}
Nov 04 01:58:10.587090 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.587001    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25f2a6a50668ca6ef956fa94a0262baaf49c791dae07cde5f8cd696a36cb3b03"
Nov 04 01:58:10.587090 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.587060    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65edf5d0-f7bhm"
Nov 04 01:58:10.599583 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.599538    6116 scope.go:122] "RemoveContainer" containerID="589c975d481f8234f9faf58c83e27b0a5a1f8f3d2dc87189ba1c69ea1cfd5f2a"
Nov 04 01:58:10.611515 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.611488    6116 scope.go:122] "RemoveContainer" containerID="7810a6f5c8c7c6c57582c3cc5da64104795d7970d7444b97fcee6c686d30c92c"
Nov 04 01:58:10.611850 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:58:10.611825    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7810a6f5c8c7c6c57582c3cc5da64104795d7970d7444b97fcee6c686d30c92c\": container with ID starting with 7810a6f5c8c7c6c57582c3cc5da64104795d7970d7444b97fcee6c686d30c92c not found: ID does not exist" containerID="7810a6f5c8c7c6c57582c3cc5da64104795d7970d7444b97fcee6c686d30c92c"
Nov 04 01:58:10.611943 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.611861    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7810a6f5c8c7c6c57582c3cc5da64104795d7970d7444b97fcee6c686d30c92c"} err="failed to get container status \"7810a6f5c8c7c6c57582c3cc5da64104795d7970d7444b97fcee6c686d30c92c\": rpc error: code = NotFound desc = could not find container \"7810a6f5c8c7c6c57582c3cc5da64104795d7970d7444b97fcee6c686d30c92c\": container with ID starting with 7810a6f5c8c7c6c57582c3cc5da64104795d7970d7444b97fcee6c686d30c92c not found: ID does not exist"
Nov 04 01:58:10.611943 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.611889    6116 scope.go:122] "RemoveContainer" containerID="589c975d481f8234f9faf58c83e27b0a5a1f8f3d2dc87189ba1c69ea1cfd5f2a"
Nov 04 01:58:10.612162 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:58:10.612141    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"589c975d481f8234f9faf58c83e27b0a5a1f8f3d2dc87189ba1c69ea1cfd5f2a\": container with ID starting with 589c975d481f8234f9faf58c83e27b0a5a1f8f3d2dc87189ba1c69ea1cfd5f2a not found: ID does not exist" containerID="589c975d481f8234f9faf58c83e27b0a5a1f8f3d2dc87189ba1c69ea1cfd5f2a"
Nov 04 01:58:10.612225 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.612169    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"589c975d481f8234f9faf58c83e27b0a5a1f8f3d2dc87189ba1c69ea1cfd5f2a"} err="failed to get container status \"589c975d481f8234f9faf58c83e27b0a5a1f8f3d2dc87189ba1c69ea1cfd5f2a\": rpc error: code = NotFound desc = could not find container \"589c975d481f8234f9faf58c83e27b0a5a1f8f3d2dc87189ba1c69ea1cfd5f2a\": container with ID starting with 589c975d481f8234f9faf58c83e27b0a5a1f8f3d2dc87189ba1c69ea1cfd5f2a not found: ID does not exist"
Nov 04 01:58:10.804039 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.804004    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"]
Nov 04 01:58:10.808863 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:10.808829    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6"]
Nov 04 01:58:11.027402 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:11.027366    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a649747c-ccad-4629-8be4-43fc827eb3f9" path="/var/lib/kubelet/pods/a649747c-ccad-4629-8be4-43fc827eb3f9/volumes"
Nov 04 01:58:11.994056 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:58:11.994017    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:58:16.451543 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:16.451508    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42"]
Nov 04 01:58:16.557428 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:16.557381    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw"]
Nov 04 01:58:18.092359 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.092325    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w"]
Nov 04 01:58:18.092829 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.092815    6116 cpu_manager.go:414] "containerMap: removing container" podUID="a649747c-ccad-4629-8be4-43fc827eb3f9" containerName="istio-proxy"
Nov 04 01:58:18.092872 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.092830    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a649747c-ccad-4629-8be4-43fc827eb3f9" containerName="istio-proxy"
Nov 04 01:58:18.092872 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.092864    6116 cpu_manager.go:414] "containerMap: removing container" podUID="6ddcfdc7-9d1d-4698-b43d-6a673d31497a" containerName="revoke-keys"
Nov 04 01:58:18.092872 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.092869    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ddcfdc7-9d1d-4698-b43d-6a673d31497a" containerName="revoke-keys"
Nov 04 01:58:18.092963 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.092932    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="a649747c-ccad-4629-8be4-43fc827eb3f9" containerName="istio-proxy"
Nov 04 01:58:18.092963 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.092943    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="a649747c-ccad-4629-8be4-43fc827eb3f9" containerName="istio-proxy"
Nov 04 01:58:18.092963 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.092950    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ddcfdc7-9d1d-4698-b43d-6a673d31497a" containerName="revoke-keys"
Nov 04 01:58:18.097748 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.097730    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w"
Nov 04 01:58:18.150425 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.150392    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:58:18.150707 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.150413    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 01:58:18.154121 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.154095    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w"]
Nov 04 01:58:18.198956 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.198920    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v74ph\" (UniqueName: \"kubernetes.io/projected/cedbd40c-2a91-4928-ab9c-a2f647d3896d-kube-api-access-v74ph\") pod \"maas-api-revoke-sub-39dbe6fd-8xr7w\" (UID: \"cedbd40c-2a91-4928-ab9c-a2f647d3896d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w"
Nov 04 01:58:18.199137 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.198991    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cedbd40c-2a91-4928-ab9c-a2f647d3896d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-39dbe6fd-8xr7w\" (UID: \"cedbd40c-2a91-4928-ab9c-a2f647d3896d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w"
Nov 04 01:58:18.199281 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.199253    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cedbd40c-2a91-4928-ab9c-a2f647d3896d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-39dbe6fd-8xr7w\" (UID: \"cedbd40c-2a91-4928-ab9c-a2f647d3896d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w"
Nov 04 01:58:18.199727 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.199705    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:58:18.200288 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.200268    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bj8b6\""
Nov 04 01:58:18.300459 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.300407    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v74ph\" (UniqueName: \"kubernetes.io/projected/cedbd40c-2a91-4928-ab9c-a2f647d3896d-kube-api-access-v74ph\") pod \"maas-api-revoke-sub-39dbe6fd-8xr7w\" (UID: \"cedbd40c-2a91-4928-ab9c-a2f647d3896d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w"
Nov 04 01:58:18.300656 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.300512    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cedbd40c-2a91-4928-ab9c-a2f647d3896d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-39dbe6fd-8xr7w\" (UID: \"cedbd40c-2a91-4928-ab9c-a2f647d3896d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w"
Nov 04 01:58:18.300656 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.300607    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cedbd40c-2a91-4928-ab9c-a2f647d3896d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-39dbe6fd-8xr7w\" (UID: \"cedbd40c-2a91-4928-ab9c-a2f647d3896d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w"
Nov 04 01:58:18.301538 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.301510    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cedbd40c-2a91-4928-ab9c-a2f647d3896d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-39dbe6fd-8xr7w\" (UID: \"cedbd40c-2a91-4928-ab9c-a2f647d3896d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w"
Nov 04 01:58:18.303178 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.303154    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cedbd40c-2a91-4928-ab9c-a2f647d3896d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-39dbe6fd-8xr7w\" (UID: \"cedbd40c-2a91-4928-ab9c-a2f647d3896d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w"
Nov 04 01:58:18.361616 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.361538    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v74ph\" (UniqueName: \"kubernetes.io/projected/cedbd40c-2a91-4928-ab9c-a2f647d3896d-kube-api-access-v74ph\") pod \"maas-api-revoke-sub-39dbe6fd-8xr7w\" (UID: \"cedbd40c-2a91-4928-ab9c-a2f647d3896d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w"
Nov 04 01:58:18.409937 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.409908    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w"
Nov 04 01:58:18.785749 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:18.785724    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w"]
Nov 04 01:58:18.786885 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:58:18.786859    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcedbd40c_2a91_4928_ab9c_a2f647d3896d.slice/crio-31192df3978d290080b9cf1c54f7efc067333c3370cc74898637124f10d1ac42 WatchSource:0}: Error finding container 31192df3978d290080b9cf1c54f7efc067333c3370cc74898637124f10d1ac42: Status 404 returned error can't find the container with id 31192df3978d290080b9cf1c54f7efc067333c3370cc74898637124f10d1ac42
Nov 04 01:58:19.650791 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:19.650757    6116 generic.go:356] "Generic (PLEG): container finished" podID="cedbd40c-2a91-4928-ab9c-a2f647d3896d" containerID="13e6b2d23ddeed6f2a69af782156791ba8efae9f95d3fb657a52581b53a9e543" exitCode=0
Nov 04 01:58:19.651226 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:19.650800    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w" event={"ID":"cedbd40c-2a91-4928-ab9c-a2f647d3896d","Type":"ContainerDied","Data":"13e6b2d23ddeed6f2a69af782156791ba8efae9f95d3fb657a52581b53a9e543"}
Nov 04 01:58:19.651226 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:19.650846    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w" event={"ID":"cedbd40c-2a91-4928-ab9c-a2f647d3896d","Type":"ContainerStarted","Data":"31192df3978d290080b9cf1c54f7efc067333c3370cc74898637124f10d1ac42"}
Nov 04 01:58:20.796216 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:20.796183    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w"
Nov 04 01:58:20.932329 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:20.932253    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cedbd40c-2a91-4928-ab9c-a2f647d3896d-kube-api-access-v74ph\" (UniqueName: \"kubernetes.io/projected/cedbd40c-2a91-4928-ab9c-a2f647d3896d-kube-api-access-v74ph\") pod \"cedbd40c-2a91-4928-ab9c-a2f647d3896d\" (UID: \"cedbd40c-2a91-4928-ab9c-a2f647d3896d\") "
Nov 04 01:58:20.932329 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:20.932302    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cedbd40c-2a91-4928-ab9c-a2f647d3896d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cedbd40c-2a91-4928-ab9c-a2f647d3896d-maas-api-openshift-ca\") pod \"cedbd40c-2a91-4928-ab9c-a2f647d3896d\" (UID: \"cedbd40c-2a91-4928-ab9c-a2f647d3896d\") "
Nov 04 01:58:20.932603 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:20.932442    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cedbd40c-2a91-4928-ab9c-a2f647d3896d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cedbd40c-2a91-4928-ab9c-a2f647d3896d-maas-api-xks-ca\") pod \"cedbd40c-2a91-4928-ab9c-a2f647d3896d\" (UID: \"cedbd40c-2a91-4928-ab9c-a2f647d3896d\") "
Nov 04 01:58:20.932759 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:20.932734    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cedbd40c-2a91-4928-ab9c-a2f647d3896d-maas-api-openshift-ca" pod "cedbd40c-2a91-4928-ab9c-a2f647d3896d" (UID: "cedbd40c-2a91-4928-ab9c-a2f647d3896d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:58:20.932879 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:20.932853    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cedbd40c-2a91-4928-ab9c-a2f647d3896d-maas-api-openshift-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:20.934412 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:20.934389    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cedbd40c-2a91-4928-ab9c-a2f647d3896d-kube-api-access-v74ph" pod "cedbd40c-2a91-4928-ab9c-a2f647d3896d" (UID: "cedbd40c-2a91-4928-ab9c-a2f647d3896d"). InnerVolumeSpecName "kube-api-access-v74ph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:20.934953 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:20.934937    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cedbd40c-2a91-4928-ab9c-a2f647d3896d-maas-api-xks-ca" pod "cedbd40c-2a91-4928-ab9c-a2f647d3896d" (UID: "cedbd40c-2a91-4928-ab9c-a2f647d3896d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:58:21.033705 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:21.033672    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v74ph\" (UniqueName: \"kubernetes.io/projected/cedbd40c-2a91-4928-ab9c-a2f647d3896d-kube-api-access-v74ph\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:21.033705 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:21.033704    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cedbd40c-2a91-4928-ab9c-a2f647d3896d-maas-api-xks-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:21.451910 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:21.451868    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42" podUID="fe49df1c-2c17-440d-8e83-3b6ec337e909" containerName="payload-processing" containerID="cri-o://e3a70a3836348e8b6bdb5d91e139a2486501599b54b66c82ee02164ddf5c2375" gracePeriod=25
Nov 04 01:58:21.558851 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:21.558811    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw" podUID="e3bcfc78-b4c1-4448-b848-a6400e8b6912" containerName="payload-pre-processing" containerID="cri-o://f283355c22bd884bfd699ffc09e08512ade7622f90b997a5fbb2f00f21ca2038" gracePeriod=25
Nov 04 01:58:21.666690 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:21.666659    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w"
Nov 04 01:58:21.666690 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:21.666673    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-39dbe6fd-8xr7w" event={"ID":"cedbd40c-2a91-4928-ab9c-a2f647d3896d","Type":"ContainerDied","Data":"31192df3978d290080b9cf1c54f7efc067333c3370cc74898637124f10d1ac42"}
Nov 04 01:58:21.666901 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:21.666760    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="31192df3978d290080b9cf1c54f7efc067333c3370cc74898637124f10d1ac42"
Nov 04 01:58:23.996259 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:23.996223    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42" podUID="fe49df1c-2c17-440d-8e83-3b6ec337e909" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:58:23.996259 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:23.996241    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw" podUID="e3bcfc78-b4c1-4448-b848-a6400e8b6912" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:58:25.227565 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.227529    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn" podUID="105b1344-7994-4f1b-b6c6-1476c606d52f" containerName="main" containerID="cri-o://c64111a1ce9bd38457d732493d29f1ea44dab5fea28fa9398ef962857da6ff84" gracePeriod=45
Nov 04 01:58:25.245515 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.245472    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp" podUID="a3dbce40-4ed1-4237-aa6c-8018e49f5e9e" containerName="main" containerID="cri-o://034829695f889e6da2d69849614067be3a55a6ffb27e92af4eba045e98a5d32b" gracePeriod=45
Nov 04 01:58:25.284919 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.284887    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp" podUID="a3dbce40-4ed1-4237-aa6c-8018e49f5e9e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": dial tcp 10.134.0.54:8000: connect: connection refused"
Nov 04 01:58:25.507012 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.506987    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:58:25.532709 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.532684    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:58:25.573739 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.573714    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-model-cache\") pod \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") "
Nov 04 01:58:25.573888 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.573787    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/105b1344-7994-4f1b-b6c6-1476c606d52f-kube-api-access-2krlb\" (UniqueName: \"kubernetes.io/projected/105b1344-7994-4f1b-b6c6-1476c606d52f-kube-api-access-2krlb\") pod \"105b1344-7994-4f1b-b6c6-1476c606d52f\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") "
Nov 04 01:58:25.573888 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.573806    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/105b1344-7994-4f1b-b6c6-1476c606d52f-tls-certs\" (UniqueName: \"kubernetes.io/secret/105b1344-7994-4f1b-b6c6-1476c606d52f-tls-certs\") pod \"105b1344-7994-4f1b-b6c6-1476c606d52f\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") "
Nov 04 01:58:25.573888 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.573831    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-tmp-dir\") pod \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") "
Nov 04 01:58:25.573888 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.573871    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-tls-certs\") pod \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") "
Nov 04 01:58:25.574107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.573896    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-model-cache\") pod \"105b1344-7994-4f1b-b6c6-1476c606d52f\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") "
Nov 04 01:58:25.574107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.573945    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-dshm\") pod \"105b1344-7994-4f1b-b6c6-1476c606d52f\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") "
Nov 04 01:58:25.574107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.573981    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-model-cache" pod "a3dbce40-4ed1-4237-aa6c-8018e49f5e9e" (UID: "a3dbce40-4ed1-4237-aa6c-8018e49f5e9e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:25.574107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.574005    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-home\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-home\") pod \"105b1344-7994-4f1b-b6c6-1476c606d52f\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") "
Nov 04 01:58:25.574107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.574030    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-dshm\") pod \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") "
Nov 04 01:58:25.574107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.574074    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-home\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-home\") pod \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") "
Nov 04 01:58:25.574107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.574105    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-tmp-dir\") pod \"105b1344-7994-4f1b-b6c6-1476c606d52f\" (UID: \"105b1344-7994-4f1b-b6c6-1476c606d52f\") "
Nov 04 01:58:25.574514 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.574114    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-tmp-dir" pod "a3dbce40-4ed1-4237-aa6c-8018e49f5e9e" (UID: "a3dbce40-4ed1-4237-aa6c-8018e49f5e9e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:25.574514 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.574178    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-kube-api-access-k7w4t\" (UniqueName: \"kubernetes.io/projected/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-kube-api-access-k7w4t\") pod \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\" (UID: \"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e\") "
Nov 04 01:58:25.574514 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.574188    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-model-cache" pod "105b1344-7994-4f1b-b6c6-1476c606d52f" (UID: "105b1344-7994-4f1b-b6c6-1476c606d52f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:25.574514 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.574282    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-home" pod "105b1344-7994-4f1b-b6c6-1476c606d52f" (UID: "105b1344-7994-4f1b-b6c6-1476c606d52f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:25.574514 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.574413    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-home" pod "a3dbce40-4ed1-4237-aa6c-8018e49f5e9e" (UID: "a3dbce40-4ed1-4237-aa6c-8018e49f5e9e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:25.574764 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.574510    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-tmp-dir" pod "105b1344-7994-4f1b-b6c6-1476c606d52f" (UID: "105b1344-7994-4f1b-b6c6-1476c606d52f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:25.574764 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.574729    6116 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-home\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:25.574764 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.574748    6116 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-home\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:25.574764 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.574760    6116 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-tmp-dir\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:25.574953 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.574773    6116 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-model-cache\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:25.574953 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.574788    6116 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-tmp-dir\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:25.574953 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.574801    6116 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-model-cache\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:25.576532 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.576510    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-dshm" pod "a3dbce40-4ed1-4237-aa6c-8018e49f5e9e" (UID: "a3dbce40-4ed1-4237-aa6c-8018e49f5e9e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:25.576699 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.576685    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/105b1344-7994-4f1b-b6c6-1476c606d52f-tls-certs" pod "105b1344-7994-4f1b-b6c6-1476c606d52f" (UID: "105b1344-7994-4f1b-b6c6-1476c606d52f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:58:25.576752 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.576705    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-dshm" pod "105b1344-7994-4f1b-b6c6-1476c606d52f" (UID: "105b1344-7994-4f1b-b6c6-1476c606d52f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:25.576752 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.576710    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-tls-certs" pod "a3dbce40-4ed1-4237-aa6c-8018e49f5e9e" (UID: "a3dbce40-4ed1-4237-aa6c-8018e49f5e9e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:58:25.576813 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.576751    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/105b1344-7994-4f1b-b6c6-1476c606d52f-kube-api-access-2krlb" pod "105b1344-7994-4f1b-b6c6-1476c606d52f" (UID: "105b1344-7994-4f1b-b6c6-1476c606d52f"). InnerVolumeSpecName "kube-api-access-2krlb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:25.576941 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.576927    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-kube-api-access-k7w4t" pod "a3dbce40-4ed1-4237-aa6c-8018e49f5e9e" (UID: "a3dbce40-4ed1-4237-aa6c-8018e49f5e9e"). InnerVolumeSpecName "kube-api-access-k7w4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:25.676174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.676134    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7w4t\" (UniqueName: \"kubernetes.io/projected/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-kube-api-access-k7w4t\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:25.676174 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.676172    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2krlb\" (UniqueName: \"kubernetes.io/projected/105b1344-7994-4f1b-b6c6-1476c606d52f-kube-api-access-2krlb\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:25.676376 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.676186    6116 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/105b1344-7994-4f1b-b6c6-1476c606d52f-tls-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:25.676376 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.676201    6116 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-tls-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:25.676376 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.676214    6116 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/105b1344-7994-4f1b-b6c6-1476c606d52f-dshm\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:25.676376 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.676227    6116 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e-dshm\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:25.691474 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.691422    6116 generic.go:356] "Generic (PLEG): container finished" podID="a3dbce40-4ed1-4237-aa6c-8018e49f5e9e" containerID="034829695f889e6da2d69849614067be3a55a6ffb27e92af4eba045e98a5d32b" exitCode=0
Nov 04 01:58:25.691641 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.691496    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"
Nov 04 01:58:25.691641 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.691514    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp" event={"ID":"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e","Type":"ContainerDied","Data":"034829695f889e6da2d69849614067be3a55a6ffb27e92af4eba045e98a5d32b"}
Nov 04 01:58:25.691641 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.691555    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp" event={"ID":"a3dbce40-4ed1-4237-aa6c-8018e49f5e9e","Type":"ContainerDied","Data":"05e027641cfc15332f94971b454ea1d2da2f127b9e1e5908b1f7b885d1189dd2"}
Nov 04 01:58:25.691641 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.691579    6116 scope.go:122] "RemoveContainer" containerID="034829695f889e6da2d69849614067be3a55a6ffb27e92af4eba045e98a5d32b"
Nov 04 01:58:25.693283 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.693075    6116 generic.go:356] "Generic (PLEG): container finished" podID="105b1344-7994-4f1b-b6c6-1476c606d52f" containerID="c64111a1ce9bd38457d732493d29f1ea44dab5fea28fa9398ef962857da6ff84" exitCode=0
Nov 04 01:58:25.693283 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.693139    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"
Nov 04 01:58:25.693283 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.693214    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn" event={"ID":"105b1344-7994-4f1b-b6c6-1476c606d52f","Type":"ContainerDied","Data":"c64111a1ce9bd38457d732493d29f1ea44dab5fea28fa9398ef962857da6ff84"}
Nov 04 01:58:25.693283 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.693256    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn" event={"ID":"105b1344-7994-4f1b-b6c6-1476c606d52f","Type":"ContainerDied","Data":"28376d1861f75f0a146615978a317aa0a46ae9e1e9ecc42817615f14d34fcf31"}
Nov 04 01:58:25.701757 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.701739    6116 scope.go:122] "RemoveContainer" containerID="034829695f889e6da2d69849614067be3a55a6ffb27e92af4eba045e98a5d32b"
Nov 04 01:58:25.702109 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:58:25.701996    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"034829695f889e6da2d69849614067be3a55a6ffb27e92af4eba045e98a5d32b\": container with ID starting with 034829695f889e6da2d69849614067be3a55a6ffb27e92af4eba045e98a5d32b not found: ID does not exist" containerID="034829695f889e6da2d69849614067be3a55a6ffb27e92af4eba045e98a5d32b"
Nov 04 01:58:25.702109 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.702023    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"034829695f889e6da2d69849614067be3a55a6ffb27e92af4eba045e98a5d32b"} err="failed to get container status \"034829695f889e6da2d69849614067be3a55a6ffb27e92af4eba045e98a5d32b\": rpc error: code = NotFound desc = could not find container \"034829695f889e6da2d69849614067be3a55a6ffb27e92af4eba045e98a5d32b\": container with ID starting with 034829695f889e6da2d69849614067be3a55a6ffb27e92af4eba045e98a5d32b not found: ID does not exist"
Nov 04 01:58:25.702109 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.702045    6116 scope.go:122] "RemoveContainer" containerID="c64111a1ce9bd38457d732493d29f1ea44dab5fea28fa9398ef962857da6ff84"
Nov 04 01:58:25.711184 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.711166    6116 scope.go:122] "RemoveContainer" containerID="c64111a1ce9bd38457d732493d29f1ea44dab5fea28fa9398ef962857da6ff84"
Nov 04 01:58:25.711435 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:58:25.711418    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c64111a1ce9bd38457d732493d29f1ea44dab5fea28fa9398ef962857da6ff84\": container with ID starting with c64111a1ce9bd38457d732493d29f1ea44dab5fea28fa9398ef962857da6ff84 not found: ID does not exist" containerID="c64111a1ce9bd38457d732493d29f1ea44dab5fea28fa9398ef962857da6ff84"
Nov 04 01:58:25.711538 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.711443    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c64111a1ce9bd38457d732493d29f1ea44dab5fea28fa9398ef962857da6ff84"} err="failed to get container status \"c64111a1ce9bd38457d732493d29f1ea44dab5fea28fa9398ef962857da6ff84\": rpc error: code = NotFound desc = could not find container \"c64111a1ce9bd38457d732493d29f1ea44dab5fea28fa9398ef962857da6ff84\": container with ID starting with c64111a1ce9bd38457d732493d29f1ea44dab5fea28fa9398ef962857da6ff84 not found: ID does not exist"
Nov 04 01:58:25.852477 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.852378    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"]
Nov 04 01:58:25.859164 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.859138    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-82706f/distinct-2-w2-82706f-kserve-7db8d86777-4wjbn"]
Nov 04 01:58:25.969524 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.969489    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"]
Nov 04 01:58:25.973600 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:25.973573    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-82706f/simulator-w2-82706f-kserve-646c6c569d-phzfp"]
Nov 04 01:58:27.027159 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:27.027122    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="105b1344-7994-4f1b-b6c6-1476c606d52f" path="/var/lib/kubelet/pods/105b1344-7994-4f1b-b6c6-1476c606d52f/volumes"
Nov 04 01:58:27.027610 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:27.027595    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a3dbce40-4ed1-4237-aa6c-8018e49f5e9e" path="/var/lib/kubelet/pods/a3dbce40-4ed1-4237-aa6c-8018e49f5e9e/volumes"
Nov 04 01:58:28.581787 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.581751    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"]
Nov 04 01:58:28.582210 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.582202    6116 cpu_manager.go:414] "containerMap: removing container" podUID="a649747c-ccad-4629-8be4-43fc827eb3f9" containerName="istio-proxy"
Nov 04 01:58:28.582254 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.582213    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a649747c-ccad-4629-8be4-43fc827eb3f9" containerName="istio-proxy"
Nov 04 01:58:28.582254 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.582223    6116 cpu_manager.go:414] "containerMap: removing container" podUID="a3dbce40-4ed1-4237-aa6c-8018e49f5e9e" containerName="main"
Nov 04 01:58:28.582254 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.582228    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3dbce40-4ed1-4237-aa6c-8018e49f5e9e" containerName="main"
Nov 04 01:58:28.582254 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.582236    6116 cpu_manager.go:414] "containerMap: removing container" podUID="105b1344-7994-4f1b-b6c6-1476c606d52f" containerName="main"
Nov 04 01:58:28.582254 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.582241    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="105b1344-7994-4f1b-b6c6-1476c606d52f" containerName="main"
Nov 04 01:58:28.582254 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.582250    6116 cpu_manager.go:414] "containerMap: removing container" podUID="cedbd40c-2a91-4928-ab9c-a2f647d3896d" containerName="revoke-keys"
Nov 04 01:58:28.582254 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.582254    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cedbd40c-2a91-4928-ab9c-a2f647d3896d" containerName="revoke-keys"
Nov 04 01:58:28.582480 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.582348    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3dbce40-4ed1-4237-aa6c-8018e49f5e9e" containerName="main"
Nov 04 01:58:28.582480 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.582355    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="105b1344-7994-4f1b-b6c6-1476c606d52f" containerName="main"
Nov 04 01:58:28.582480 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.582362    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="cedbd40c-2a91-4928-ab9c-a2f647d3896d" containerName="revoke-keys"
Nov 04 01:58:28.587201 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.587179    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.608254 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.608219    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33450851-a5a5-4567-8e86-b85e478d3bce-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.608389 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.608265    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.608389 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.608336    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.608389 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.608379    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33450851-a5a5-4567-8e86-b85e478d3bce-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.608515 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.608480    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.608555 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.608534    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.608622 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.608603    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78kcf\" (UniqueName: \"kubernetes.io/projected/33450851-a5a5-4567-8e86-b85e478d3bce-kube-api-access-78kcf\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.608656 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.608648    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33450851-a5a5-4567-8e86-b85e478d3bce-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.608740 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.608728    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.631440 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.631411    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"]
Nov 04 01:58:28.635800 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.635780    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.687665 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.687630    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"]
Nov 04 01:58:28.710299 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.710272    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.710499 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.710322    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.710499 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.710369    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.710499 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.710385    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.710499 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.710421    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78kcf\" (UniqueName: \"kubernetes.io/projected/33450851-a5a5-4567-8e86-b85e478d3bce-kube-api-access-78kcf\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.710871 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.710683    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33450851-a5a5-4567-8e86-b85e478d3bce-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.710871 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.710742    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.710871 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.710750    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.710871 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.710811    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.710871 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.710752    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.710871 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.710868    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.711170 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.710908    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33450851-a5a5-4567-8e86-b85e478d3bce-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.711170 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.710952    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.711170 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.710978    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.711170 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.711006    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.711170 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.711013    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.711170 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.711021    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.711474 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.711377    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.711474 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.711408    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jccb\" (UniqueName: \"kubernetes.io/projected/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-kube-api-access-9jccb\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.711580 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.711470    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.711580 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.711508    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33450851-a5a5-4567-8e86-b85e478d3bce-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.711675 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.711663    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.712120 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.712101    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33450851-a5a5-4567-8e86-b85e478d3bce-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.713345 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.713322    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.713568 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.713552    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33450851-a5a5-4567-8e86-b85e478d3bce-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.741921 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.741560    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"]
Nov 04 01:58:28.771034 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.771002    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33450851-a5a5-4567-8e86-b85e478d3bce-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.771034 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.771021    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78kcf\" (UniqueName: \"kubernetes.io/projected/33450851-a5a5-4567-8e86-b85e478d3bce-kube-api-access-78kcf\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.812620 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.812582    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.812832 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.812644    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.812832 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.812717    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.812832 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.812749    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.812832 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.812799    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.813051 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.812949    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.813051 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.812994    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.813051 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.813027    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jccb\" (UniqueName: \"kubernetes.io/projected/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-kube-api-access-9jccb\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.813200 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.813056    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.813200 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.813174    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.813604 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.813522    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.813604 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.813598    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.813834 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.813812    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.814273 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.814244    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.815684 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.815662    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.815950 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.815933    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.875061 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.874982    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.875412 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.875386    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jccb\" (UniqueName: \"kubernetes.io/projected/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-kube-api-access-9jccb\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:28.899858 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.899832    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:28.948717 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:28.948688    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:29.326077 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:29.325954    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"]
Nov 04 01:58:29.331568 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:29.331534    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:58:29.331653 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:29.331601    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:58:29.331653 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:29.331629    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:58:29.378933 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:29.378907    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"]
Nov 04 01:58:29.379507 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:58:29.379483    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ce4bd82_51e8_4b93_a2fd_db2e721cbcf9.slice/crio-07f9a10c5860ffd730d2b165c9972a15fbb96ad72810e4f4944247309ca1f312 WatchSource:0}: Error finding container 07f9a10c5860ffd730d2b165c9972a15fbb96ad72810e4f4944247309ca1f312: Status 404 returned error can't find the container with id 07f9a10c5860ffd730d2b165c9972a15fbb96ad72810e4f4944247309ca1f312
Nov 04 01:58:29.381832 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:29.381799    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:58:29.381899 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:29.381866    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:58:29.381935 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:29.381900    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:58:29.727530 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:29.727497    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs" event={"ID":"33450851-a5a5-4567-8e86-b85e478d3bce","Type":"ContainerStarted","Data":"9654d248063b4f68adbffea61d62cb508e1e31c2793c2ce605785500104d2231"}
Nov 04 01:58:29.728050 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:29.728024    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs" event={"ID":"33450851-a5a5-4567-8e86-b85e478d3bce","Type":"ContainerStarted","Data":"5190b092236edc44f90f6e9ec95bebd8a4e4e8f0a5623135e8a271fcce4f3e04"}
Nov 04 01:58:29.731476 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:29.729375    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw" event={"ID":"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9","Type":"ContainerStarted","Data":"e24a2ee4a39e17208e12c50eb72e7ada6a6af7ccb8846d13f250205779b7c5ab"}
Nov 04 01:58:29.731476 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:29.729423    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw" event={"ID":"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9","Type":"ContainerStarted","Data":"07f9a10c5860ffd730d2b165c9972a15fbb96ad72810e4f4944247309ca1f312"}
Nov 04 01:58:29.900604 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:29.900548    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:29.949575 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:29.949542    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:29.994546 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:29.990678    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs" podStartSLOduration=1.9906610219999998 podStartE2EDuration="1.990661022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:58:29.852802023 +0000 UTC m=+1394.974079552" watchObservedRunningTime="2026-11-04 01:58:29.990661022 +0000 UTC m=+1395.111938549"
Nov 04 01:58:30.902238 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:30.902193    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:58:30.902684 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:30.902250    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs" podUID="33450851-a5a5-4567-8e86-b85e478d3bce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Nov 04 01:58:30.986486 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:30.968138    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 04 01:58:30.986486 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:30.968199    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw" podUID="6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 04 01:58:31.900277 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:31.900243    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:58:31.900548 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:31.900523    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs" podUID="33450851-a5a5-4567-8e86-b85e478d3bce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Nov 04 01:58:31.951476 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:31.949834    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 04 01:58:31.951476 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:31.949895    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw" podUID="6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 04 01:58:32.902239 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:32.901703    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:58:32.902239 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:32.901753    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs" podUID="33450851-a5a5-4567-8e86-b85e478d3bce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Nov 04 01:58:32.954485 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:32.950731    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 04 01:58:32.954485 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:32.950788    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw" podUID="6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 04 01:58:33.900855 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:33.900817    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:58:33.901207 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:33.901173    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs" podUID="33450851-a5a5-4567-8e86-b85e478d3bce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Nov 04 01:58:33.955372 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:33.955335    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 04 01:58:33.964714 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:33.964674    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw" podUID="6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 04 01:58:33.996897 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:33.996854    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw" podUID="e3bcfc78-b4c1-4448-b848-a6400e8b6912" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:58:33.997610 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:33.997577    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42" podUID="fe49df1c-2c17-440d-8e83-3b6ec337e909" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:58:34.900871 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:34.900831    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:58:34.901058 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:34.900891    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs" podUID="33450851-a5a5-4567-8e86-b85e478d3bce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Nov 04 01:58:34.950482 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:34.949820    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 04 01:58:34.950482 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:34.949877    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw" podUID="6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 04 01:58:35.901109 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:35.901067    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Nov 04 01:58:35.901554 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:35.901139    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs" podUID="33450851-a5a5-4567-8e86-b85e478d3bce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Nov 04 01:58:35.949404 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:35.949368    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 04 01:58:35.949589 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:35.949435    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw" podUID="6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 04 01:58:37.942595 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:37.942560    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:58:37.942982 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:37.942611    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs" podUID="33450851-a5a5-4567-8e86-b85e478d3bce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:58:37.991683 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:37.991639    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:58:37.991900 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:37.991711    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw" podUID="6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:58:38.059968 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:38.059932    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:38.060382 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:38.060355    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:38.061197 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:38.061175    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:38.224950 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:38.224841    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw" podStartSLOduration=10.224820556 podStartE2EDuration="10.224820556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:58:29.989886122 +0000 UTC m=+1395.111163650" watchObservedRunningTime="2026-11-04 01:58:38.224820556 +0000 UTC m=+1403.346098078"
Nov 04 01:58:38.229727 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:38.229695    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:38.230033 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:38.230018    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:38.231677 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:38.231645    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:38.592473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:38.592373    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:58:41.723809 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.723737    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42"
Nov 04 01:58:41.813964 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.813938    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw"
Nov 04 01:58:41.825079 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.825057    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe49df1c-2c17-440d-8e83-3b6ec337e909-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe49df1c-2c17-440d-8e83-3b6ec337e909-plugins-config-volume\") pod \"fe49df1c-2c17-440d-8e83-3b6ec337e909\" (UID: \"fe49df1c-2c17-440d-8e83-3b6ec337e909\") "
Nov 04 01:58:41.825173 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.825138    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe49df1c-2c17-440d-8e83-3b6ec337e909-kube-api-access-szcrp\" (UniqueName: \"kubernetes.io/projected/fe49df1c-2c17-440d-8e83-3b6ec337e909-kube-api-access-szcrp\") pod \"fe49df1c-2c17-440d-8e83-3b6ec337e909\" (UID: \"fe49df1c-2c17-440d-8e83-3b6ec337e909\") "
Nov 04 01:58:41.825443 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.825422    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe49df1c-2c17-440d-8e83-3b6ec337e909-plugins-config-volume" pod "fe49df1c-2c17-440d-8e83-3b6ec337e909" (UID: "fe49df1c-2c17-440d-8e83-3b6ec337e909"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:58:41.825637 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.825624    6116 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe49df1c-2c17-440d-8e83-3b6ec337e909-plugins-config-volume\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:41.827303 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.827284    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe49df1c-2c17-440d-8e83-3b6ec337e909-kube-api-access-szcrp" pod "fe49df1c-2c17-440d-8e83-3b6ec337e909" (UID: "fe49df1c-2c17-440d-8e83-3b6ec337e909"). InnerVolumeSpecName "kube-api-access-szcrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:41.854355 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.854324    6116 generic.go:356] "Generic (PLEG): container finished" podID="e3bcfc78-b4c1-4448-b848-a6400e8b6912" containerID="f283355c22bd884bfd699ffc09e08512ade7622f90b997a5fbb2f00f21ca2038" exitCode=0
Nov 04 01:58:41.854541 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.854362    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw"
Nov 04 01:58:41.854541 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.854410    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw" event={"ID":"e3bcfc78-b4c1-4448-b848-a6400e8b6912","Type":"ContainerDied","Data":"f283355c22bd884bfd699ffc09e08512ade7622f90b997a5fbb2f00f21ca2038"}
Nov 04 01:58:41.854541 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.854500    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw" event={"ID":"e3bcfc78-b4c1-4448-b848-a6400e8b6912","Type":"ContainerDied","Data":"57af1f65f2a65f9708157c4ddee375b9d4d656bacd9190220817da091dcd5263"}
Nov 04 01:58:41.854541 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.854525    6116 scope.go:122] "RemoveContainer" containerID="f283355c22bd884bfd699ffc09e08512ade7622f90b997a5fbb2f00f21ca2038"
Nov 04 01:58:41.856095 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.856071    6116 generic.go:356] "Generic (PLEG): container finished" podID="fe49df1c-2c17-440d-8e83-3b6ec337e909" containerID="e3a70a3836348e8b6bdb5d91e139a2486501599b54b66c82ee02164ddf5c2375" exitCode=0
Nov 04 01:58:41.856195 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.856112    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42"
Nov 04 01:58:41.856195 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.856116    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42" event={"ID":"fe49df1c-2c17-440d-8e83-3b6ec337e909","Type":"ContainerDied","Data":"e3a70a3836348e8b6bdb5d91e139a2486501599b54b66c82ee02164ddf5c2375"}
Nov 04 01:58:41.856195 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.856155    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42" event={"ID":"fe49df1c-2c17-440d-8e83-3b6ec337e909","Type":"ContainerDied","Data":"3166368cd7a37495c28e7299ce97de0e9092c79cae6679a95caaeb458b903470"}
Nov 04 01:58:41.864631 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.864603    6116 scope.go:122] "RemoveContainer" containerID="f283355c22bd884bfd699ffc09e08512ade7622f90b997a5fbb2f00f21ca2038"
Nov 04 01:58:41.864890 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:58:41.864869    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f283355c22bd884bfd699ffc09e08512ade7622f90b997a5fbb2f00f21ca2038\": container with ID starting with f283355c22bd884bfd699ffc09e08512ade7622f90b997a5fbb2f00f21ca2038 not found: ID does not exist" containerID="f283355c22bd884bfd699ffc09e08512ade7622f90b997a5fbb2f00f21ca2038"
Nov 04 01:58:41.864980 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.864895    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f283355c22bd884bfd699ffc09e08512ade7622f90b997a5fbb2f00f21ca2038"} err="failed to get container status \"f283355c22bd884bfd699ffc09e08512ade7622f90b997a5fbb2f00f21ca2038\": rpc error: code = NotFound desc = could not find container \"f283355c22bd884bfd699ffc09e08512ade7622f90b997a5fbb2f00f21ca2038\": container with ID starting with f283355c22bd884bfd699ffc09e08512ade7622f90b997a5fbb2f00f21ca2038 not found: ID does not exist"
Nov 04 01:58:41.864980 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.864911    6116 scope.go:122] "RemoveContainer" containerID="e3a70a3836348e8b6bdb5d91e139a2486501599b54b66c82ee02164ddf5c2375"
Nov 04 01:58:41.873874 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.873856    6116 scope.go:122] "RemoveContainer" containerID="e3a70a3836348e8b6bdb5d91e139a2486501599b54b66c82ee02164ddf5c2375"
Nov 04 01:58:41.874177 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:58:41.874159    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3a70a3836348e8b6bdb5d91e139a2486501599b54b66c82ee02164ddf5c2375\": container with ID starting with e3a70a3836348e8b6bdb5d91e139a2486501599b54b66c82ee02164ddf5c2375 not found: ID does not exist" containerID="e3a70a3836348e8b6bdb5d91e139a2486501599b54b66c82ee02164ddf5c2375"
Nov 04 01:58:41.874241 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.874189    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3a70a3836348e8b6bdb5d91e139a2486501599b54b66c82ee02164ddf5c2375"} err="failed to get container status \"e3a70a3836348e8b6bdb5d91e139a2486501599b54b66c82ee02164ddf5c2375\": rpc error: code = NotFound desc = could not find container \"e3a70a3836348e8b6bdb5d91e139a2486501599b54b66c82ee02164ddf5c2375\": container with ID starting with e3a70a3836348e8b6bdb5d91e139a2486501599b54b66c82ee02164ddf5c2375 not found: ID does not exist"
Nov 04 01:58:41.925980 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.925950    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3bcfc78-b4c1-4448-b848-a6400e8b6912-kube-api-access-gbmfd\" (UniqueName: \"kubernetes.io/projected/e3bcfc78-b4c1-4448-b848-a6400e8b6912-kube-api-access-gbmfd\") pod \"e3bcfc78-b4c1-4448-b848-a6400e8b6912\" (UID: \"e3bcfc78-b4c1-4448-b848-a6400e8b6912\") "
Nov 04 01:58:41.926130 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.925997    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3bcfc78-b4c1-4448-b848-a6400e8b6912-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3bcfc78-b4c1-4448-b848-a6400e8b6912-plugins-config-volume\") pod \"e3bcfc78-b4c1-4448-b848-a6400e8b6912\" (UID: \"e3bcfc78-b4c1-4448-b848-a6400e8b6912\") "
Nov 04 01:58:41.926316 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.926304    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szcrp\" (UniqueName: \"kubernetes.io/projected/fe49df1c-2c17-440d-8e83-3b6ec337e909-kube-api-access-szcrp\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:41.926353 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.926335    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3bcfc78-b4c1-4448-b848-a6400e8b6912-plugins-config-volume" pod "e3bcfc78-b4c1-4448-b848-a6400e8b6912" (UID: "e3bcfc78-b4c1-4448-b848-a6400e8b6912"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:58:41.928139 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:41.928118    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3bcfc78-b4c1-4448-b848-a6400e8b6912-kube-api-access-gbmfd" pod "e3bcfc78-b4c1-4448-b848-a6400e8b6912" (UID: "e3bcfc78-b4c1-4448-b848-a6400e8b6912"). InnerVolumeSpecName "kube-api-access-gbmfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:41.995285 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:58:41.995189    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:58:42.027702 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:42.027670    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbmfd\" (UniqueName: \"kubernetes.io/projected/e3bcfc78-b4c1-4448-b848-a6400e8b6912-kube-api-access-gbmfd\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:42.027702 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:42.027696    6116 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3bcfc78-b4c1-4448-b848-a6400e8b6912-plugins-config-volume\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:42.065181 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:42.065152    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42"]
Nov 04 01:58:42.069235 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:42.069205    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42"]
Nov 04 01:58:42.274642 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:42.274557    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw"]
Nov 04 01:58:42.280714 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:42.280685    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw"]
Nov 04 01:58:43.027530 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.027495    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3bcfc78-b4c1-4448-b848-a6400e8b6912" path="/var/lib/kubelet/pods/e3bcfc78-b4c1-4448-b848-a6400e8b6912/volumes"
Nov 04 01:58:43.027925 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.027896    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe49df1c-2c17-440d-8e83-3b6ec337e909" path="/var/lib/kubelet/pods/fe49df1c-2c17-440d-8e83-3b6ec337e909/volumes"
Nov 04 01:58:43.726370 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.726339    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"]
Nov 04 01:58:43.726829 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.726816    6116 cpu_manager.go:414] "containerMap: removing container" podUID="fe49df1c-2c17-440d-8e83-3b6ec337e909" containerName="payload-processing"
Nov 04 01:58:43.726895 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.726831    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe49df1c-2c17-440d-8e83-3b6ec337e909" containerName="payload-processing"
Nov 04 01:58:43.726895 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.726851    6116 cpu_manager.go:414] "containerMap: removing container" podUID="e3bcfc78-b4c1-4448-b848-a6400e8b6912" containerName="payload-pre-processing"
Nov 04 01:58:43.726895 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.726856    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3bcfc78-b4c1-4448-b848-a6400e8b6912" containerName="payload-pre-processing"
Nov 04 01:58:43.726991 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.726951    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe49df1c-2c17-440d-8e83-3b6ec337e909" containerName="payload-processing"
Nov 04 01:58:43.726991 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.726959    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3bcfc78-b4c1-4448-b848-a6400e8b6912" containerName="payload-pre-processing"
Nov 04 01:58:43.731707 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.731686    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.838063 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.838033    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"]
Nov 04 01:58:43.845263 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.845235    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.845443 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.845285    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.845443 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.845305    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.845443 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.845378    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bggps\" (UniqueName: \"kubernetes.io/projected/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-kube-api-access-bggps\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.845641 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.845539    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.845641 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.845606    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.845740 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.845697    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.845790 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.845736    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.845849 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.845793    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.947132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.947096    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.947284 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.947152    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.947284 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.947191    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.947284 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.947264    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.947518 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.947322    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.947518 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.947349    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.947518 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.947480    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bggps\" (UniqueName: \"kubernetes.io/projected/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-kube-api-access-bggps\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.947678 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.947554    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.947678 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.947584    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-credential-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.947678 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.947618    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.947678 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.947660    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-workload-socket\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.947988 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.947965    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-workload-certs\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.948229 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.948150    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istiod-ca-cert\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.948435 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.948377    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-data\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.949771 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.949752    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-envoy\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:43.949983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:43.949966    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-podinfo\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:44.008892 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:44.008812    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-token\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:44.008892 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:44.008869    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bggps\" (UniqueName: \"kubernetes.io/projected/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-kube-api-access-bggps\") pod \"e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") " pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:44.044700 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:44.044674    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:44.469971 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:44.469946    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"]
Nov 04 01:58:44.471196 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:58:44.471171    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaa8ad32_cb35_48e2_a4b9_5dc397ce962b.slice/crio-c4d9a3a0faf85a48282e810003e5b0fe24763b4f02cfbf8723e3fa343f8b196c WatchSource:0}: Error finding container c4d9a3a0faf85a48282e810003e5b0fe24763b4f02cfbf8723e3fa343f8b196c: Status 404 returned error can't find the container with id c4d9a3a0faf85a48282e810003e5b0fe24763b4f02cfbf8723e3fa343f8b196c
Nov 04 01:58:44.473484 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:44.473435    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:58:44.473605 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:44.473521    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:58:44.473605 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:44.473551    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:58:44.877695 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:44.877607    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf" event={"ID":"daa8ad32-cb35-48e2-a4b9-5dc397ce962b","Type":"ContainerStarted","Data":"e567e3d05103aebc32186aaf917e5b15e26415851f3417b55ad05bc6bfd18e54"}
Nov 04 01:58:44.877695 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:44.877661    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf" event={"ID":"daa8ad32-cb35-48e2-a4b9-5dc397ce962b","Type":"ContainerStarted","Data":"c4d9a3a0faf85a48282e810003e5b0fe24763b4f02cfbf8723e3fa343f8b196c"}
Nov 04 01:58:44.999094 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:44.999041    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf" podStartSLOduration=1.9990259529999999 podStartE2EDuration="1.999025953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:58:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:58:44.996804492 +0000 UTC m=+1410.118082019" watchObservedRunningTime="2026-11-04 01:58:44.999025953 +0000 UTC m=+1410.120303478"
Nov 04 01:58:45.045238 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:45.045202    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:45.047072 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:45.047044    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 04 01:58:45.047191 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:45.047103    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf" podUID="daa8ad32-cb35-48e2-a4b9-5dc397ce962b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 04 01:58:46.056475 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:46.053889    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 04 01:58:46.056475 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:46.053953    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf" podUID="daa8ad32-cb35-48e2-a4b9-5dc397ce962b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 04 01:58:47.066480 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:47.062584    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 04 01:58:47.066480 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:47.062670    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf" podUID="daa8ad32-cb35-48e2-a4b9-5dc397ce962b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 04 01:58:48.063893 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:48.063850    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 04 01:58:48.064156 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:48.064123    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf" podUID="daa8ad32-cb35-48e2-a4b9-5dc397ce962b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 04 01:58:49.056473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:49.045732    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 04 01:58:49.056473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:49.045804    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf" podUID="daa8ad32-cb35-48e2-a4b9-5dc397ce962b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 04 01:58:50.045278 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:50.045242    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 04 01:58:50.045496 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:50.045293    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf" podUID="daa8ad32-cb35-48e2-a4b9-5dc397ce962b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 04 01:58:51.049016 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:51.048988    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:51.049574 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:51.049400    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:51.050113 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:51.050094    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:51.510108 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:51.510075    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"]
Nov 04 01:58:51.511497 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:51.511443    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw" podUID="6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" containerName="istio-proxy" containerID="cri-o://e24a2ee4a39e17208e12c50eb72e7ada6a6af7ccb8846d13f250205779b7c5ab" gracePeriod=30
Nov 04 01:58:51.518341 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:51.518309    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"]
Nov 04 01:58:51.520903 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:51.520879    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"]
Nov 04 01:58:51.521077 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:51.521059    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k" podUID="30488fc4-bfaf-414c-9dcd-6e250398b8dc" containerName="istio-proxy" containerID="cri-o://c0665c99a090db1e0d631877d792d6995d18434c3c1e8ecc01fe0e128e817861" gracePeriod=30
Nov 04 01:58:51.537377 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:51.533233    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"]
Nov 04 01:58:51.537377 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:51.533552    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs" podUID="33450851-a5a5-4567-8e86-b85e478d3bce" containerName="istio-proxy" containerID="cri-o://9654d248063b4f68adbffea61d62cb508e1e31c2793c2ce605785500104d2231" gracePeriod=30
Nov 04 01:58:51.922413 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:51.922381    6116 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf" secret="" err="secret \"e2e-worker-w2-82706f-openshift-default-dockercfg-v862d\" not found"
Nov 04 01:58:52.928231 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:52.928176    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf" podUID="daa8ad32-cb35-48e2-a4b9-5dc397ce962b" containerName="istio-proxy" containerID="cri-o://e567e3d05103aebc32186aaf917e5b15e26415851f3417b55ad05bc6bfd18e54" gracePeriod=30
Nov 04 01:58:53.048770 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:53.048735    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx"]
Nov 04 01:58:53.063282 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:53.063247    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:58:53.063437 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:53.063308    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs" podUID="33450851-a5a5-4567-8e86-b85e478d3bce" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:58:53.098207 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:53.098178    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5487d7d8-s98rx"]
Nov 04 01:58:53.232277 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:53.232191    6116 patch_prober.go:29] interesting pod/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Nov 04 01:58:53.232277 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:53.232251    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw" podUID="6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:58:55.027241 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:55.027209    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2fb79b6e-5c0e-491b-ae5b-337ecd99901e" path="/var/lib/kubelet/pods/2fb79b6e-5c0e-491b-ae5b-337ecd99901e/volumes"
Nov 04 01:58:56.802728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.802660    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:56.835302 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.835275    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:58:56.846715 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.846684    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:56.898436 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898407    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-workload-certs\") pod \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") "
Nov 04 01:58:56.898615 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898492    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-workload-certs\") pod \"33450851-a5a5-4567-8e86-b85e478d3bce\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") "
Nov 04 01:58:56.898615 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898521    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33450851-a5a5-4567-8e86-b85e478d3bce-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33450851-a5a5-4567-8e86-b85e478d3bce-istiod-ca-cert\") pod \"33450851-a5a5-4567-8e86-b85e478d3bce\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") "
Nov 04 01:58:56.898615 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898536    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-istio-data\") pod \"33450851-a5a5-4567-8e86-b85e478d3bce\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") "
Nov 04 01:58:56.898615 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898553    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istiod-ca-cert\") pod \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") "
Nov 04 01:58:56.898615 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898578    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-data\") pod \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") "
Nov 04 01:58:56.898889 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898677    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-credential-socket\") pod \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") "
Nov 04 01:58:56.898889 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898722    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-envoy\") pod \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") "
Nov 04 01:58:56.898889 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898724    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-workload-certs" pod "6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" (UID: "6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.898889 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898752    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30488fc4-bfaf-414c-9dcd-6e250398b8dc-kube-api-access-bt5lv\" (UniqueName: \"kubernetes.io/projected/30488fc4-bfaf-414c-9dcd-6e250398b8dc-kube-api-access-bt5lv\") pod \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") "
Nov 04 01:58:56.898889 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898786    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-workload-socket\") pod \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") "
Nov 04 01:58:56.898889 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898829    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-workload-socket\") pod \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") "
Nov 04 01:58:56.898889 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898838    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-workload-certs" pod "33450851-a5a5-4567-8e86-b85e478d3bce" (UID: "33450851-a5a5-4567-8e86-b85e478d3bce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.898889 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898842    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-data" pod "6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" (UID: "6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.898889 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898854    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-istio-data" pod "33450851-a5a5-4567-8e86-b85e478d3bce" (UID: "33450851-a5a5-4567-8e86-b85e478d3bce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.898889 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898872    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-data\") pod \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") "
Nov 04 01:58:56.899393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898901    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-token\" (UniqueName: \"kubernetes.io/projected/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-token\") pod \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") "
Nov 04 01:58:56.899393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898923    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istiod-ca-cert" pod "6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" (UID: "6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:58:56.899393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898933    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/33450851-a5a5-4567-8e86-b85e478d3bce-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33450851-a5a5-4567-8e86-b85e478d3bce-istio-podinfo\") pod \"33450851-a5a5-4567-8e86-b85e478d3bce\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") "
Nov 04 01:58:56.899393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898969    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istiod-ca-cert\") pod \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") "
Nov 04 01:58:56.899393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.899004    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-istio-envoy\") pod \"33450851-a5a5-4567-8e86-b85e478d3bce\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") "
Nov 04 01:58:56.899393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.899035    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-envoy\") pod \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") "
Nov 04 01:58:56.899393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.899064    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-credential-socket\") pod \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") "
Nov 04 01:58:56.899393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.899114    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33450851-a5a5-4567-8e86-b85e478d3bce-istio-token\" (UniqueName: \"kubernetes.io/projected/33450851-a5a5-4567-8e86-b85e478d3bce-istio-token\") pod \"33450851-a5a5-4567-8e86-b85e478d3bce\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") "
Nov 04 01:58:56.899393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.899160    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-credential-socket\") pod \"33450851-a5a5-4567-8e86-b85e478d3bce\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") "
Nov 04 01:58:56.899393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.899193    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-token\" (UniqueName: \"kubernetes.io/projected/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-token\") pod \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") "
Nov 04 01:58:56.899393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.899263    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-kube-api-access-9jccb\" (UniqueName: \"kubernetes.io/projected/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-kube-api-access-9jccb\") pod \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") "
Nov 04 01:58:56.899393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.899292    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-podinfo\") pod \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") "
Nov 04 01:58:56.899393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.899324    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-workload-socket\") pod \"33450851-a5a5-4567-8e86-b85e478d3bce\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") "
Nov 04 01:58:56.899393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.899366    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-podinfo\") pod \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\" (UID: \"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9\") "
Nov 04 01:58:56.900268 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.899420    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33450851-a5a5-4567-8e86-b85e478d3bce-kube-api-access-78kcf\" (UniqueName: \"kubernetes.io/projected/33450851-a5a5-4567-8e86-b85e478d3bce-kube-api-access-78kcf\") pod \"33450851-a5a5-4567-8e86-b85e478d3bce\" (UID: \"33450851-a5a5-4567-8e86-b85e478d3bce\") "
Nov 04 01:58:56.900268 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.899465    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-workload-certs\") pod \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\" (UID: \"30488fc4-bfaf-414c-9dcd-6e250398b8dc\") "
Nov 04 01:58:56.900268 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898968    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33450851-a5a5-4567-8e86-b85e478d3bce-istiod-ca-cert" pod "33450851-a5a5-4567-8e86-b85e478d3bce" (UID: "33450851-a5a5-4567-8e86-b85e478d3bce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:58:56.900268 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.898978    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-credential-socket" pod "6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" (UID: "6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.900268 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.899105    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-workload-socket" pod "6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" (UID: "6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.900268 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.899291    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-data" pod "30488fc4-bfaf-414c-9dcd-6e250398b8dc" (UID: "30488fc4-bfaf-414c-9dcd-6e250398b8dc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.900268 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.899304    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-workload-socket" pod "30488fc4-bfaf-414c-9dcd-6e250398b8dc" (UID: "30488fc4-bfaf-414c-9dcd-6e250398b8dc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.900268 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.899485    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-credential-socket" pod "30488fc4-bfaf-414c-9dcd-6e250398b8dc" (UID: "30488fc4-bfaf-414c-9dcd-6e250398b8dc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.900268 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.899739    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istiod-ca-cert" pod "30488fc4-bfaf-414c-9dcd-6e250398b8dc" (UID: "30488fc4-bfaf-414c-9dcd-6e250398b8dc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:58:56.901152 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.900966    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-workload-socket" pod "33450851-a5a5-4567-8e86-b85e478d3bce" (UID: "33450851-a5a5-4567-8e86-b85e478d3bce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.901730 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.901495    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-workload-certs" pod "30488fc4-bfaf-414c-9dcd-6e250398b8dc" (UID: "30488fc4-bfaf-414c-9dcd-6e250398b8dc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.901730 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.901672    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-credential-socket" pod "33450851-a5a5-4567-8e86-b85e478d3bce" (UID: "33450851-a5a5-4567-8e86-b85e478d3bce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.901991 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.901948    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.901991 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.901971    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.901991 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.901986    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.902234 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.901998    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.902234 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.902043    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.902234 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.902054    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33450851-a5a5-4567-8e86-b85e478d3bce-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.902234 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.902066    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.902234 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.902067    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-token" pod "30488fc4-bfaf-414c-9dcd-6e250398b8dc" (UID: "30488fc4-bfaf-414c-9dcd-6e250398b8dc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:56.902234 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.902079    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.902234 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.902092    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.902234 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.902106    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.902234 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.902112    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-token" pod "6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" (UID: "6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:56.902234 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.902118    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.902234 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.902153    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.902234 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.902170    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.902234 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.902180    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.902234 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.902188    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:56.902234 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.902131    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33450851-a5a5-4567-8e86-b85e478d3bce-istio-podinfo" pod "33450851-a5a5-4567-8e86-b85e478d3bce" (UID: "33450851-a5a5-4567-8e86-b85e478d3bce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:58:56.903166 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.902610    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30488fc4-bfaf-414c-9dcd-6e250398b8dc-kube-api-access-bt5lv" pod "30488fc4-bfaf-414c-9dcd-6e250398b8dc" (UID: "30488fc4-bfaf-414c-9dcd-6e250398b8dc"). InnerVolumeSpecName "kube-api-access-bt5lv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:56.903166 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.903011    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-istio-envoy" pod "33450851-a5a5-4567-8e86-b85e478d3bce" (UID: "33450851-a5a5-4567-8e86-b85e478d3bce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.903166 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.903078    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-envoy" pod "6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" (UID: "6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.903516 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.903428    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-kube-api-access-9jccb" pod "6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" (UID: "6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9"). InnerVolumeSpecName "kube-api-access-9jccb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:56.903516 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.903428    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-envoy" pod "30488fc4-bfaf-414c-9dcd-6e250398b8dc" (UID: "30488fc4-bfaf-414c-9dcd-6e250398b8dc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:56.904195 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.904178    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33450851-a5a5-4567-8e86-b85e478d3bce-kube-api-access-78kcf" pod "33450851-a5a5-4567-8e86-b85e478d3bce" (UID: "33450851-a5a5-4567-8e86-b85e478d3bce"). InnerVolumeSpecName "kube-api-access-78kcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:56.904296 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.904279    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-podinfo" pod "6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" (UID: "6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:58:56.904529 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.904505    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33450851-a5a5-4567-8e86-b85e478d3bce-istio-token" pod "33450851-a5a5-4567-8e86-b85e478d3bce" (UID: "33450851-a5a5-4567-8e86-b85e478d3bce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:56.904910 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.904894    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-podinfo" pod "30488fc4-bfaf-414c-9dcd-6e250398b8dc" (UID: "30488fc4-bfaf-414c-9dcd-6e250398b8dc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:58:56.954577 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.954543    6116 generic.go:356] "Generic (PLEG): container finished" podID="33450851-a5a5-4567-8e86-b85e478d3bce" containerID="9654d248063b4f68adbffea61d62cb508e1e31c2793c2ce605785500104d2231" exitCode=0
Nov 04 01:58:56.954739 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.954591    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"
Nov 04 01:58:56.954739 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.954621    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs" event={"ID":"33450851-a5a5-4567-8e86-b85e478d3bce","Type":"ContainerDied","Data":"9654d248063b4f68adbffea61d62cb508e1e31c2793c2ce605785500104d2231"}
Nov 04 01:58:56.954739 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.954700    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs" event={"ID":"33450851-a5a5-4567-8e86-b85e478d3bce","Type":"ContainerDied","Data":"5190b092236edc44f90f6e9ec95bebd8a4e4e8f0a5623135e8a271fcce4f3e04"}
Nov 04 01:58:56.954739 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.954730    6116 scope.go:122] "RemoveContainer" containerID="9654d248063b4f68adbffea61d62cb508e1e31c2793c2ce605785500104d2231"
Nov 04 01:58:56.956395 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.956373    6116 generic.go:356] "Generic (PLEG): container finished" podID="30488fc4-bfaf-414c-9dcd-6e250398b8dc" containerID="c0665c99a090db1e0d631877d792d6995d18434c3c1e8ecc01fe0e128e817861" exitCode=0
Nov 04 01:58:56.956531 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.956422    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"
Nov 04 01:58:56.956531 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.956485    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k" event={"ID":"30488fc4-bfaf-414c-9dcd-6e250398b8dc","Type":"ContainerDied","Data":"c0665c99a090db1e0d631877d792d6995d18434c3c1e8ecc01fe0e128e817861"}
Nov 04 01:58:56.956628 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.956533    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k" event={"ID":"30488fc4-bfaf-414c-9dcd-6e250398b8dc","Type":"ContainerDied","Data":"a23193b18b8e624d0e66b0dbd33b4a97b0af9b7492539ebbf7c6a5eaba82552a"}
Nov 04 01:58:56.958066 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.958046    6116 generic.go:356] "Generic (PLEG): container finished" podID="6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" containerID="e24a2ee4a39e17208e12c50eb72e7ada6a6af7ccb8846d13f250205779b7c5ab" exitCode=0
Nov 04 01:58:56.958181 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.958099    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"
Nov 04 01:58:56.958181 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.958121    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw" event={"ID":"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9","Type":"ContainerDied","Data":"e24a2ee4a39e17208e12c50eb72e7ada6a6af7ccb8846d13f250205779b7c5ab"}
Nov 04 01:58:56.958181 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.958162    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw" event={"ID":"6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9","Type":"ContainerDied","Data":"07f9a10c5860ffd730d2b165c9972a15fbb96ad72810e4f4944247309ca1f312"}
Nov 04 01:58:56.967287 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.967195    6116 scope.go:122] "RemoveContainer" containerID="9654d248063b4f68adbffea61d62cb508e1e31c2793c2ce605785500104d2231"
Nov 04 01:58:56.967576 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:58:56.967555    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9654d248063b4f68adbffea61d62cb508e1e31c2793c2ce605785500104d2231\": container with ID starting with 9654d248063b4f68adbffea61d62cb508e1e31c2793c2ce605785500104d2231 not found: ID does not exist" containerID="9654d248063b4f68adbffea61d62cb508e1e31c2793c2ce605785500104d2231"
Nov 04 01:58:56.967681 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.967585    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9654d248063b4f68adbffea61d62cb508e1e31c2793c2ce605785500104d2231"} err="failed to get container status \"9654d248063b4f68adbffea61d62cb508e1e31c2793c2ce605785500104d2231\": rpc error: code = NotFound desc = could not find container \"9654d248063b4f68adbffea61d62cb508e1e31c2793c2ce605785500104d2231\": container with ID starting with 9654d248063b4f68adbffea61d62cb508e1e31c2793c2ce605785500104d2231 not found: ID does not exist"
Nov 04 01:58:56.967681 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.967601    6116 scope.go:122] "RemoveContainer" containerID="c0665c99a090db1e0d631877d792d6995d18434c3c1e8ecc01fe0e128e817861"
Nov 04 01:58:56.977534 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.977519    6116 scope.go:122] "RemoveContainer" containerID="b2abc865032b5f7b32978de46dfcf3c5cc8fcf9664faaf52d2c89b58bdd23f2e"
Nov 04 01:58:56.986537 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.986517    6116 scope.go:122] "RemoveContainer" containerID="c0665c99a090db1e0d631877d792d6995d18434c3c1e8ecc01fe0e128e817861"
Nov 04 01:58:56.986817 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:58:56.986794    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0665c99a090db1e0d631877d792d6995d18434c3c1e8ecc01fe0e128e817861\": container with ID starting with c0665c99a090db1e0d631877d792d6995d18434c3c1e8ecc01fe0e128e817861 not found: ID does not exist" containerID="c0665c99a090db1e0d631877d792d6995d18434c3c1e8ecc01fe0e128e817861"
Nov 04 01:58:56.986900 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.986825    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0665c99a090db1e0d631877d792d6995d18434c3c1e8ecc01fe0e128e817861"} err="failed to get container status \"c0665c99a090db1e0d631877d792d6995d18434c3c1e8ecc01fe0e128e817861\": rpc error: code = NotFound desc = could not find container \"c0665c99a090db1e0d631877d792d6995d18434c3c1e8ecc01fe0e128e817861\": container with ID starting with c0665c99a090db1e0d631877d792d6995d18434c3c1e8ecc01fe0e128e817861 not found: ID does not exist"
Nov 04 01:58:56.986900 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.986843    6116 scope.go:122] "RemoveContainer" containerID="b2abc865032b5f7b32978de46dfcf3c5cc8fcf9664faaf52d2c89b58bdd23f2e"
Nov 04 01:58:56.987131 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:58:56.987112    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2abc865032b5f7b32978de46dfcf3c5cc8fcf9664faaf52d2c89b58bdd23f2e\": container with ID starting with b2abc865032b5f7b32978de46dfcf3c5cc8fcf9664faaf52d2c89b58bdd23f2e not found: ID does not exist" containerID="b2abc865032b5f7b32978de46dfcf3c5cc8fcf9664faaf52d2c89b58bdd23f2e"
Nov 04 01:58:56.987208 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.987140    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2abc865032b5f7b32978de46dfcf3c5cc8fcf9664faaf52d2c89b58bdd23f2e"} err="failed to get container status \"b2abc865032b5f7b32978de46dfcf3c5cc8fcf9664faaf52d2c89b58bdd23f2e\": rpc error: code = NotFound desc = could not find container \"b2abc865032b5f7b32978de46dfcf3c5cc8fcf9664faaf52d2c89b58bdd23f2e\": container with ID starting with b2abc865032b5f7b32978de46dfcf3c5cc8fcf9664faaf52d2c89b58bdd23f2e not found: ID does not exist"
Nov 04 01:58:56.987208 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.987163    6116 scope.go:122] "RemoveContainer" containerID="e24a2ee4a39e17208e12c50eb72e7ada6a6af7ccb8846d13f250205779b7c5ab"
Nov 04 01:58:56.995654 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.995636    6116 scope.go:122] "RemoveContainer" containerID="e24a2ee4a39e17208e12c50eb72e7ada6a6af7ccb8846d13f250205779b7c5ab"
Nov 04 01:58:56.995890 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:58:56.995875    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e24a2ee4a39e17208e12c50eb72e7ada6a6af7ccb8846d13f250205779b7c5ab\": container with ID starting with e24a2ee4a39e17208e12c50eb72e7ada6a6af7ccb8846d13f250205779b7c5ab not found: ID does not exist" containerID="e24a2ee4a39e17208e12c50eb72e7ada6a6af7ccb8846d13f250205779b7c5ab"
Nov 04 01:58:56.995953 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:56.995897    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e24a2ee4a39e17208e12c50eb72e7ada6a6af7ccb8846d13f250205779b7c5ab"} err="failed to get container status \"e24a2ee4a39e17208e12c50eb72e7ada6a6af7ccb8846d13f250205779b7c5ab\": rpc error: code = NotFound desc = could not find container \"e24a2ee4a39e17208e12c50eb72e7ada6a6af7ccb8846d13f250205779b7c5ab\": container with ID starting with e24a2ee4a39e17208e12c50eb72e7ada6a6af7ccb8846d13f250205779b7c5ab not found: ID does not exist"
Nov 04 01:58:57.003507 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:57.003486    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:57.003578 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:57.003508    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33450851-a5a5-4567-8e86-b85e478d3bce-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:57.003578 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:57.003519    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33450851-a5a5-4567-8e86-b85e478d3bce-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:57.003578 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:57.003527    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:57.003578 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:57.003536    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33450851-a5a5-4567-8e86-b85e478d3bce-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:57.003578 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:57.003544    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:57.003578 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:57.003552    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jccb\" (UniqueName: \"kubernetes.io/projected/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-kube-api-access-9jccb\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:57.003578 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:57.003561    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30488fc4-bfaf-414c-9dcd-6e250398b8dc-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:57.003578 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:57.003569    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:57.003578 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:57.003576    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78kcf\" (UniqueName: \"kubernetes.io/projected/33450851-a5a5-4567-8e86-b85e478d3bce-kube-api-access-78kcf\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:57.003832 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:57.003586    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:57.003832 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:57.003594    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bt5lv\" (UniqueName: \"kubernetes.io/projected/30488fc4-bfaf-414c-9dcd-6e250398b8dc-kube-api-access-bt5lv\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:57.153718 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:57.153686    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"]
Nov 04 01:58:57.157718 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:57.157693    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs"]
Nov 04 01:58:57.335259 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:57.335222    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"]
Nov 04 01:58:57.340648 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:57.340616    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k"]
Nov 04 01:58:57.449279 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:57.449201    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"]
Nov 04 01:58:57.453684 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:57.453652    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw"]
Nov 04 01:58:58.182605 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.182580    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:58.215695 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.215624    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-token\" (UniqueName: \"kubernetes.io/projected/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-token\") pod \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") "
Nov 04 01:58:58.215695 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.215668    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-kube-api-access-bggps\" (UniqueName: \"kubernetes.io/projected/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-kube-api-access-bggps\") pod \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") "
Nov 04 01:58:58.215898 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.215696    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-workload-socket\") pod \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") "
Nov 04 01:58:58.215898 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.215752    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-credential-socket\") pod \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") "
Nov 04 01:58:58.215898 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.215803    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-envoy\") pod \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") "
Nov 04 01:58:58.216044 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.215917    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-data\") pod \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") "
Nov 04 01:58:58.216044 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.215953    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-podinfo\") pod \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") "
Nov 04 01:58:58.216044 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.215986    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-workload-certs\") pod \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") "
Nov 04 01:58:58.216193 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.216040    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-workload-socket" pod "daa8ad32-cb35-48e2-a4b9-5dc397ce962b" (UID: "daa8ad32-cb35-48e2-a4b9-5dc397ce962b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:58.216193 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.216072    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-credential-socket" pod "daa8ad32-cb35-48e2-a4b9-5dc397ce962b" (UID: "daa8ad32-cb35-48e2-a4b9-5dc397ce962b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:58.216193 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.216083    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istiod-ca-cert\") pod \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\" (UID: \"daa8ad32-cb35-48e2-a4b9-5dc397ce962b\") "
Nov 04 01:58:58.216415 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.216377    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istiod-ca-cert" pod "daa8ad32-cb35-48e2-a4b9-5dc397ce962b" (UID: "daa8ad32-cb35-48e2-a4b9-5dc397ce962b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:58:58.217031 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.216631    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-workload-certs" pod "daa8ad32-cb35-48e2-a4b9-5dc397ce962b" (UID: "daa8ad32-cb35-48e2-a4b9-5dc397ce962b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:58.217031 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.216998    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-data" pod "daa8ad32-cb35-48e2-a4b9-5dc397ce962b" (UID: "daa8ad32-cb35-48e2-a4b9-5dc397ce962b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:58.217218 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.217114    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:58.217218 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.217136    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:58.217218 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.217151    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:58.217218 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.217169    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:58.217218 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.217183    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:58.218257 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.218195    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-kube-api-access-bggps" pod "daa8ad32-cb35-48e2-a4b9-5dc397ce962b" (UID: "daa8ad32-cb35-48e2-a4b9-5dc397ce962b"). InnerVolumeSpecName "kube-api-access-bggps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:58.218257 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.218207    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-token" pod "daa8ad32-cb35-48e2-a4b9-5dc397ce962b" (UID: "daa8ad32-cb35-48e2-a4b9-5dc397ce962b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:58:58.218409 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.218335    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-podinfo" pod "daa8ad32-cb35-48e2-a4b9-5dc397ce962b" (UID: "daa8ad32-cb35-48e2-a4b9-5dc397ce962b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:58:58.218976 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.218959    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-envoy" pod "daa8ad32-cb35-48e2-a4b9-5dc397ce962b" (UID: "daa8ad32-cb35-48e2-a4b9-5dc397ce962b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:58:58.318256 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.318220    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:58.318256 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.318257    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bggps\" (UniqueName: \"kubernetes.io/projected/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-kube-api-access-bggps\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:58.318481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.318274    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:58.318481 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.318286    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daa8ad32-cb35-48e2-a4b9-5dc397ce962b-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:58:58.975146 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.975111    6116 generic.go:356] "Generic (PLEG): container finished" podID="daa8ad32-cb35-48e2-a4b9-5dc397ce962b" containerID="e567e3d05103aebc32186aaf917e5b15e26415851f3417b55ad05bc6bfd18e54" exitCode=0
Nov 04 01:58:58.975300 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.975157    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"
Nov 04 01:58:58.975300 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.975186    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf" event={"ID":"daa8ad32-cb35-48e2-a4b9-5dc397ce962b","Type":"ContainerDied","Data":"e567e3d05103aebc32186aaf917e5b15e26415851f3417b55ad05bc6bfd18e54"}
Nov 04 01:58:58.975300 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.975236    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf" event={"ID":"daa8ad32-cb35-48e2-a4b9-5dc397ce962b","Type":"ContainerDied","Data":"c4d9a3a0faf85a48282e810003e5b0fe24763b4f02cfbf8723e3fa343f8b196c"}
Nov 04 01:58:58.975300 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.975258    6116 scope.go:122] "RemoveContainer" containerID="e567e3d05103aebc32186aaf917e5b15e26415851f3417b55ad05bc6bfd18e54"
Nov 04 01:58:58.985783 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.985761    6116 scope.go:122] "RemoveContainer" containerID="e567e3d05103aebc32186aaf917e5b15e26415851f3417b55ad05bc6bfd18e54"
Nov 04 01:58:58.986076 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:58:58.986054    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e567e3d05103aebc32186aaf917e5b15e26415851f3417b55ad05bc6bfd18e54\": container with ID starting with e567e3d05103aebc32186aaf917e5b15e26415851f3417b55ad05bc6bfd18e54 not found: ID does not exist" containerID="e567e3d05103aebc32186aaf917e5b15e26415851f3417b55ad05bc6bfd18e54"
Nov 04 01:58:58.986142 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:58.986084    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e567e3d05103aebc32186aaf917e5b15e26415851f3417b55ad05bc6bfd18e54"} err="failed to get container status \"e567e3d05103aebc32186aaf917e5b15e26415851f3417b55ad05bc6bfd18e54\": rpc error: code = NotFound desc = could not find container \"e567e3d05103aebc32186aaf917e5b15e26415851f3417b55ad05bc6bfd18e54\": container with ID starting with e567e3d05103aebc32186aaf917e5b15e26415851f3417b55ad05bc6bfd18e54 not found: ID does not exist"
Nov 04 01:58:59.026237 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:59.026204    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30488fc4-bfaf-414c-9dcd-6e250398b8dc" path="/var/lib/kubelet/pods/30488fc4-bfaf-414c-9dcd-6e250398b8dc/volumes"
Nov 04 01:58:59.026816 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:59.026799    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33450851-a5a5-4567-8e86-b85e478d3bce" path="/var/lib/kubelet/pods/33450851-a5a5-4567-8e86-b85e478d3bce/volumes"
Nov 04 01:58:59.027265 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:59.027251    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" path="/var/lib/kubelet/pods/6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9/volumes"
Nov 04 01:58:59.188469 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:59.188420    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"]
Nov 04 01:58:59.238090 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:58:59.238012    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf"]
Nov 04 01:59:01.027087 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:01.027049    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="daa8ad32-cb35-48e2-a4b9-5dc397ce962b" path="/var/lib/kubelet/pods/daa8ad32-cb35-48e2-a4b9-5dc397ce962b/volumes"
Nov 04 01:59:11.604408 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:11.604374    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"]
Nov 04 01:59:11.604802 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:11.604617    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q" podUID="dad0a1c4-88b2-40ee-9a4a-8997fc743448" containerName="istio-proxy" containerID="cri-o://ccdbf2ba8bdbb20e844ccebe6c48db078ac7742903ae99ce741835afd07f99d6" gracePeriod=30
Nov 04 01:59:11.653702 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:11.653652    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"]
Nov 04 01:59:11.653890 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:11.653870    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5" podUID="a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" containerName="istio-proxy" containerID="cri-o://87cd9d4f681210cb3f0c4636c5bd2a400341aac763374c98977023d3ec084454" gracePeriod=30
Nov 04 01:59:11.655983 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:11.655957    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"]
Nov 04 01:59:11.656192 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:11.656146    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m" podUID="2e4522dd-634e-4890-8ba5-67676e1575a9" containerName="istio-proxy" containerID="cri-o://0f7725dd358d69003ee9dda123aeb545c2d4c2bcb28ebf09214d388797e9f459" gracePeriod=30
Nov 04 01:59:11.704163 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:11.704125    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"]
Nov 04 01:59:11.704423 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:11.704381    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74" podUID="fde313ca-17ff-426b-814f-f27b28fd8023" containerName="istio-proxy" containerID="cri-o://be908c8cbfe3f2be2f0be9f6ea745a15c6645afd4d820d15e40b302eac2c5dcf" gracePeriod=30
Nov 04 01:59:11.990706 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:59:11.990671    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:59:15.363973 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:15.363943    6116 scope.go:122] "RemoveContainer" containerID="a107736f1f94cda6ddb67679c9575a0556043a82a013a7e872451668a22c6cef"
Nov 04 01:59:15.373060 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:15.373038    6116 scope.go:122] "RemoveContainer" containerID="4849ed0e3ef8d0fa8bcdc445f80a484d29e3a3b845a3489c1b82e82ba2f621cb"
Nov 04 01:59:15.381560 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:15.381538    6116 scope.go:122] "RemoveContainer" containerID="40dc710bc642789a8f5c2d9d8898bf80652eb822b7c7325539ceb0bbd710a717"
Nov 04 01:59:16.915619 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:16.915585    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:59:16.960543 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:16.960519    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:59:17.025131 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.025107    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-workload-socket\") pod \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") "
Nov 04 01:59:17.025286 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.025154    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istiod-ca-cert\") pod \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") "
Nov 04 01:59:17.025286 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.025184    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-workload-certs\") pod \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") "
Nov 04 01:59:17.025286 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.025211    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-envoy\") pod \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") "
Nov 04 01:59:17.025286 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.025253    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-token\" (UniqueName: \"kubernetes.io/projected/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-token\") pod \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") "
Nov 04 01:59:17.025286 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.025283    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dad0a1c4-88b2-40ee-9a4a-8997fc743448-kube-api-access-db9kl\" (UniqueName: \"kubernetes.io/projected/dad0a1c4-88b2-40ee-9a4a-8997fc743448-kube-api-access-db9kl\") pod \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") "
Nov 04 01:59:17.025763 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.025378    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-credential-socket\") pod \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") "
Nov 04 01:59:17.025763 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.025406    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-podinfo\") pod \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") "
Nov 04 01:59:17.025763 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.025627    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-workload-certs" pod "dad0a1c4-88b2-40ee-9a4a-8997fc743448" (UID: "dad0a1c4-88b2-40ee-9a4a-8997fc743448"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.025763 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.025658    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-data\") pod \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\" (UID: \"dad0a1c4-88b2-40ee-9a4a-8997fc743448\") "
Nov 04 01:59:17.025763 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.025718    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istiod-ca-cert" pod "dad0a1c4-88b2-40ee-9a4a-8997fc743448" (UID: "dad0a1c4-88b2-40ee-9a4a-8997fc743448"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:59:17.026014 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.025984    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-credential-socket" pod "dad0a1c4-88b2-40ee-9a4a-8997fc743448" (UID: "dad0a1c4-88b2-40ee-9a4a-8997fc743448"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.026986 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.026085    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.026986 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.026110    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.026986 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.026123    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.026986 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.026177    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-data" pod "dad0a1c4-88b2-40ee-9a4a-8997fc743448" (UID: "dad0a1c4-88b2-40ee-9a4a-8997fc743448"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.027559 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.027518    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-workload-socket" pod "dad0a1c4-88b2-40ee-9a4a-8997fc743448" (UID: "dad0a1c4-88b2-40ee-9a4a-8997fc743448"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.028013 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.027992    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-podinfo" pod "dad0a1c4-88b2-40ee-9a4a-8997fc743448" (UID: "dad0a1c4-88b2-40ee-9a4a-8997fc743448"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:59:17.028143 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.028126    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-token" pod "dad0a1c4-88b2-40ee-9a4a-8997fc743448" (UID: "dad0a1c4-88b2-40ee-9a4a-8997fc743448"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:59:17.028622 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.028603    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-envoy" pod "dad0a1c4-88b2-40ee-9a4a-8997fc743448" (UID: "dad0a1c4-88b2-40ee-9a4a-8997fc743448"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.028704 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.028677    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dad0a1c4-88b2-40ee-9a4a-8997fc743448-kube-api-access-db9kl" pod "dad0a1c4-88b2-40ee-9a4a-8997fc743448" (UID: "dad0a1c4-88b2-40ee-9a4a-8997fc743448"). InnerVolumeSpecName "kube-api-access-db9kl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:59:17.030152 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.030137    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:59:17.034045 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.034030    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:59:17.088539 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.088502    6116 generic.go:356] "Generic (PLEG): container finished" podID="a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" containerID="87cd9d4f681210cb3f0c4636c5bd2a400341aac763374c98977023d3ec084454" exitCode=0
Nov 04 01:59:17.088728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.088558    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"
Nov 04 01:59:17.088728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.088555    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5" event={"ID":"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd","Type":"ContainerDied","Data":"87cd9d4f681210cb3f0c4636c5bd2a400341aac763374c98977023d3ec084454"}
Nov 04 01:59:17.088728 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.088693    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5" event={"ID":"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd","Type":"ContainerDied","Data":"de5db28a8c7e96faf852602670391e91fd4d3bdabf429508e0db815191e0f325"}
Nov 04 01:59:17.088885 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.088729    6116 scope.go:122] "RemoveContainer" containerID="87cd9d4f681210cb3f0c4636c5bd2a400341aac763374c98977023d3ec084454"
Nov 04 01:59:17.090254 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.090235    6116 generic.go:356] "Generic (PLEG): container finished" podID="fde313ca-17ff-426b-814f-f27b28fd8023" containerID="be908c8cbfe3f2be2f0be9f6ea745a15c6645afd4d820d15e40b302eac2c5dcf" exitCode=0
Nov 04 01:59:17.090369 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.090288    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"
Nov 04 01:59:17.090369 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.090329    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74" event={"ID":"fde313ca-17ff-426b-814f-f27b28fd8023","Type":"ContainerDied","Data":"be908c8cbfe3f2be2f0be9f6ea745a15c6645afd4d820d15e40b302eac2c5dcf"}
Nov 04 01:59:17.090369 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.090366    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74" event={"ID":"fde313ca-17ff-426b-814f-f27b28fd8023","Type":"ContainerDied","Data":"96d6f23edbdb36cd87a04dd1ceec34572c5a3f019e57a8e32905e2214dd05061"}
Nov 04 01:59:17.091859 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.091838    6116 generic.go:356] "Generic (PLEG): container finished" podID="2e4522dd-634e-4890-8ba5-67676e1575a9" containerID="0f7725dd358d69003ee9dda123aeb545c2d4c2bcb28ebf09214d388797e9f459" exitCode=0
Nov 04 01:59:17.091961 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.091889    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"
Nov 04 01:59:17.091961 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.091921    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m" event={"ID":"2e4522dd-634e-4890-8ba5-67676e1575a9","Type":"ContainerDied","Data":"0f7725dd358d69003ee9dda123aeb545c2d4c2bcb28ebf09214d388797e9f459"}
Nov 04 01:59:17.092055 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.091961    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m" event={"ID":"2e4522dd-634e-4890-8ba5-67676e1575a9","Type":"ContainerDied","Data":"3bbe4b9bd7ada669fd51976ac0b6105e1208856915ca897de8e72f74ebed7fd5"}
Nov 04 01:59:17.093584 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.093559    6116 generic.go:356] "Generic (PLEG): container finished" podID="dad0a1c4-88b2-40ee-9a4a-8997fc743448" containerID="ccdbf2ba8bdbb20e844ccebe6c48db078ac7742903ae99ce741835afd07f99d6" exitCode=0
Nov 04 01:59:17.093691 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.093585    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q" event={"ID":"dad0a1c4-88b2-40ee-9a4a-8997fc743448","Type":"ContainerDied","Data":"ccdbf2ba8bdbb20e844ccebe6c48db078ac7742903ae99ce741835afd07f99d6"}
Nov 04 01:59:17.093691 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.093604    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"
Nov 04 01:59:17.093691 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.093620    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q" event={"ID":"dad0a1c4-88b2-40ee-9a4a-8997fc743448","Type":"ContainerDied","Data":"270afcb333e90d3a0d198a57616d60b6467a607240a11490ff9028902a8f68ca"}
Nov 04 01:59:17.098992 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.098973    6116 scope.go:122] "RemoveContainer" containerID="87cd9d4f681210cb3f0c4636c5bd2a400341aac763374c98977023d3ec084454"
Nov 04 01:59:17.099284 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:59:17.099260    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87cd9d4f681210cb3f0c4636c5bd2a400341aac763374c98977023d3ec084454\": container with ID starting with 87cd9d4f681210cb3f0c4636c5bd2a400341aac763374c98977023d3ec084454 not found: ID does not exist" containerID="87cd9d4f681210cb3f0c4636c5bd2a400341aac763374c98977023d3ec084454"
Nov 04 01:59:17.099382 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.099289    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87cd9d4f681210cb3f0c4636c5bd2a400341aac763374c98977023d3ec084454"} err="failed to get container status \"87cd9d4f681210cb3f0c4636c5bd2a400341aac763374c98977023d3ec084454\": rpc error: code = NotFound desc = could not find container \"87cd9d4f681210cb3f0c4636c5bd2a400341aac763374c98977023d3ec084454\": container with ID starting with 87cd9d4f681210cb3f0c4636c5bd2a400341aac763374c98977023d3ec084454 not found: ID does not exist"
Nov 04 01:59:17.099382 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.099305    6116 scope.go:122] "RemoveContainer" containerID="be908c8cbfe3f2be2f0be9f6ea745a15c6645afd4d820d15e40b302eac2c5dcf"
Nov 04 01:59:17.109173 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.109158    6116 scope.go:122] "RemoveContainer" containerID="be908c8cbfe3f2be2f0be9f6ea745a15c6645afd4d820d15e40b302eac2c5dcf"
Nov 04 01:59:17.109488 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:59:17.109463    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be908c8cbfe3f2be2f0be9f6ea745a15c6645afd4d820d15e40b302eac2c5dcf\": container with ID starting with be908c8cbfe3f2be2f0be9f6ea745a15c6645afd4d820d15e40b302eac2c5dcf not found: ID does not exist" containerID="be908c8cbfe3f2be2f0be9f6ea745a15c6645afd4d820d15e40b302eac2c5dcf"
Nov 04 01:59:17.109595 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.109493    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be908c8cbfe3f2be2f0be9f6ea745a15c6645afd4d820d15e40b302eac2c5dcf"} err="failed to get container status \"be908c8cbfe3f2be2f0be9f6ea745a15c6645afd4d820d15e40b302eac2c5dcf\": rpc error: code = NotFound desc = could not find container \"be908c8cbfe3f2be2f0be9f6ea745a15c6645afd4d820d15e40b302eac2c5dcf\": container with ID starting with be908c8cbfe3f2be2f0be9f6ea745a15c6645afd4d820d15e40b302eac2c5dcf not found: ID does not exist"
Nov 04 01:59:17.109595 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.109510    6116 scope.go:122] "RemoveContainer" containerID="0f7725dd358d69003ee9dda123aeb545c2d4c2bcb28ebf09214d388797e9f459"
Nov 04 01:59:17.119030 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.119003    6116 scope.go:122] "RemoveContainer" containerID="0f7725dd358d69003ee9dda123aeb545c2d4c2bcb28ebf09214d388797e9f459"
Nov 04 01:59:17.119312 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:59:17.119291    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f7725dd358d69003ee9dda123aeb545c2d4c2bcb28ebf09214d388797e9f459\": container with ID starting with 0f7725dd358d69003ee9dda123aeb545c2d4c2bcb28ebf09214d388797e9f459 not found: ID does not exist" containerID="0f7725dd358d69003ee9dda123aeb545c2d4c2bcb28ebf09214d388797e9f459"
Nov 04 01:59:17.119393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.119318    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f7725dd358d69003ee9dda123aeb545c2d4c2bcb28ebf09214d388797e9f459"} err="failed to get container status \"0f7725dd358d69003ee9dda123aeb545c2d4c2bcb28ebf09214d388797e9f459\": rpc error: code = NotFound desc = could not find container \"0f7725dd358d69003ee9dda123aeb545c2d4c2bcb28ebf09214d388797e9f459\": container with ID starting with 0f7725dd358d69003ee9dda123aeb545c2d4c2bcb28ebf09214d388797e9f459 not found: ID does not exist"
Nov 04 01:59:17.119393 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.119339    6116 scope.go:122] "RemoveContainer" containerID="ccdbf2ba8bdbb20e844ccebe6c48db078ac7742903ae99ce741835afd07f99d6"
Nov 04 01:59:17.126557 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.126535    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-data\") pod \"2e4522dd-634e-4890-8ba5-67676e1575a9\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") "
Nov 04 01:59:17.126651 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.126573    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fde313ca-17ff-426b-814f-f27b28fd8023-kube-api-access-cvxk8\" (UniqueName: \"kubernetes.io/projected/fde313ca-17ff-426b-814f-f27b28fd8023-kube-api-access-cvxk8\") pod \"fde313ca-17ff-426b-814f-f27b28fd8023\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") "
Nov 04 01:59:17.126651 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.126619    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-workload-certs\") pod \"fde313ca-17ff-426b-814f-f27b28fd8023\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") "
Nov 04 01:59:17.126771 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.126742    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-istio-data\") pod \"fde313ca-17ff-426b-814f-f27b28fd8023\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") "
Nov 04 01:59:17.126823 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.126800    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-podinfo\") pod \"2e4522dd-634e-4890-8ba5-67676e1575a9\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") "
Nov 04 01:59:17.126880 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.126826    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-data" pod "2e4522dd-634e-4890-8ba5-67676e1575a9" (UID: "2e4522dd-634e-4890-8ba5-67676e1575a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.126880 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.126833    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-credential-socket\") pod \"2e4522dd-634e-4890-8ba5-67676e1575a9\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") "
Nov 04 01:59:17.126976 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.126880    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fde313ca-17ff-426b-814f-f27b28fd8023-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fde313ca-17ff-426b-814f-f27b28fd8023-istiod-ca-cert\") pod \"fde313ca-17ff-426b-814f-f27b28fd8023\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") "
Nov 04 01:59:17.126976 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.126908    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-credential-socket\") pod \"fde313ca-17ff-426b-814f-f27b28fd8023\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") "
Nov 04 01:59:17.126976 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.126939    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fde313ca-17ff-426b-814f-f27b28fd8023-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fde313ca-17ff-426b-814f-f27b28fd8023-istio-podinfo\") pod \"fde313ca-17ff-426b-814f-f27b28fd8023\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") "
Nov 04 01:59:17.127132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.126993    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-workload-socket\") pod \"2e4522dd-634e-4890-8ba5-67676e1575a9\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") "
Nov 04 01:59:17.127132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127028    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-credential-socket" pod "2e4522dd-634e-4890-8ba5-67676e1575a9" (UID: "2e4522dd-634e-4890-8ba5-67676e1575a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.127132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127039    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e4522dd-634e-4890-8ba5-67676e1575a9-kube-api-access-9wwh6\" (UniqueName: \"kubernetes.io/projected/2e4522dd-634e-4890-8ba5-67676e1575a9-kube-api-access-9wwh6\") pod \"2e4522dd-634e-4890-8ba5-67676e1575a9\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") "
Nov 04 01:59:17.127132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127051    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-istio-data" pod "fde313ca-17ff-426b-814f-f27b28fd8023" (UID: "fde313ca-17ff-426b-814f-f27b28fd8023"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.127132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127077    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-workload-certs\") pod \"2e4522dd-634e-4890-8ba5-67676e1575a9\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") "
Nov 04 01:59:17.127132 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127104    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-token\" (UniqueName: \"kubernetes.io/projected/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-token\") pod \"2e4522dd-634e-4890-8ba5-67676e1575a9\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") "
Nov 04 01:59:17.127609 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127138    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-envoy\") pod \"2e4522dd-634e-4890-8ba5-67676e1575a9\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") "
Nov 04 01:59:17.127609 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127170    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-workload-socket\") pod \"fde313ca-17ff-426b-814f-f27b28fd8023\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") "
Nov 04 01:59:17.127609 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127210    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2e4522dd-634e-4890-8ba5-67676e1575a9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e4522dd-634e-4890-8ba5-67676e1575a9-istiod-ca-cert\") pod \"2e4522dd-634e-4890-8ba5-67676e1575a9\" (UID: \"2e4522dd-634e-4890-8ba5-67676e1575a9\") "
Nov 04 01:59:17.127609 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127240    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fde313ca-17ff-426b-814f-f27b28fd8023-istio-token\" (UniqueName: \"kubernetes.io/projected/fde313ca-17ff-426b-814f-f27b28fd8023-istio-token\") pod \"fde313ca-17ff-426b-814f-f27b28fd8023\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") "
Nov 04 01:59:17.127609 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127278    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-istio-envoy\") pod \"fde313ca-17ff-426b-814f-f27b28fd8023\" (UID: \"fde313ca-17ff-426b-814f-f27b28fd8023\") "
Nov 04 01:59:17.127609 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127167    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-workload-certs" pod "fde313ca-17ff-426b-814f-f27b28fd8023" (UID: "fde313ca-17ff-426b-814f-f27b28fd8023"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.127926 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127800    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.127926 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127821    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.127926 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127835    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.127926 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127849    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.127926 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127861    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.127926 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127874    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.127926 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127887    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dad0a1c4-88b2-40ee-9a4a-8997fc743448-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.127926 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127900    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.127926 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127914    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db9kl\" (UniqueName: \"kubernetes.io/projected/dad0a1c4-88b2-40ee-9a4a-8997fc743448-kube-api-access-db9kl\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.128389 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.128108    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e4522dd-634e-4890-8ba5-67676e1575a9-istiod-ca-cert" pod "2e4522dd-634e-4890-8ba5-67676e1575a9" (UID: "2e4522dd-634e-4890-8ba5-67676e1575a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:59:17.128389 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127262    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fde313ca-17ff-426b-814f-f27b28fd8023-istiod-ca-cert" pod "fde313ca-17ff-426b-814f-f27b28fd8023" (UID: "fde313ca-17ff-426b-814f-f27b28fd8023"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:59:17.128389 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127287    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-workload-socket" pod "2e4522dd-634e-4890-8ba5-67676e1575a9" (UID: "2e4522dd-634e-4890-8ba5-67676e1575a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.128389 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127360    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-credential-socket" pod "fde313ca-17ff-426b-814f-f27b28fd8023" (UID: "fde313ca-17ff-426b-814f-f27b28fd8023"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.128389 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.127509    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-workload-certs" pod "2e4522dd-634e-4890-8ba5-67676e1575a9" (UID: "2e4522dd-634e-4890-8ba5-67676e1575a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.128389 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.128287    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.128746 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.128432    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-workload-socket" pod "fde313ca-17ff-426b-814f-f27b28fd8023" (UID: "fde313ca-17ff-426b-814f-f27b28fd8023"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.129213 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.129191    6116 scope.go:122] "RemoveContainer" containerID="ccdbf2ba8bdbb20e844ccebe6c48db078ac7742903ae99ce741835afd07f99d6"
Nov 04 01:59:17.129562 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:59:17.129540    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccdbf2ba8bdbb20e844ccebe6c48db078ac7742903ae99ce741835afd07f99d6\": container with ID starting with ccdbf2ba8bdbb20e844ccebe6c48db078ac7742903ae99ce741835afd07f99d6 not found: ID does not exist" containerID="ccdbf2ba8bdbb20e844ccebe6c48db078ac7742903ae99ce741835afd07f99d6"
Nov 04 01:59:17.129628 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.129573    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccdbf2ba8bdbb20e844ccebe6c48db078ac7742903ae99ce741835afd07f99d6"} err="failed to get container status \"ccdbf2ba8bdbb20e844ccebe6c48db078ac7742903ae99ce741835afd07f99d6\": rpc error: code = NotFound desc = could not find container \"ccdbf2ba8bdbb20e844ccebe6c48db078ac7742903ae99ce741835afd07f99d6\": container with ID starting with ccdbf2ba8bdbb20e844ccebe6c48db078ac7742903ae99ce741835afd07f99d6 not found: ID does not exist"
Nov 04 01:59:17.129695 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.129674    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fde313ca-17ff-426b-814f-f27b28fd8023-istio-podinfo" pod "fde313ca-17ff-426b-814f-f27b28fd8023" (UID: "fde313ca-17ff-426b-814f-f27b28fd8023"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:59:17.129799 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.129782    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-envoy" pod "2e4522dd-634e-4890-8ba5-67676e1575a9" (UID: "2e4522dd-634e-4890-8ba5-67676e1575a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.130658 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.130626    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-istio-envoy" pod "fde313ca-17ff-426b-814f-f27b28fd8023" (UID: "fde313ca-17ff-426b-814f-f27b28fd8023"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.130658 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.130625    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fde313ca-17ff-426b-814f-f27b28fd8023-kube-api-access-cvxk8" pod "fde313ca-17ff-426b-814f-f27b28fd8023" (UID: "fde313ca-17ff-426b-814f-f27b28fd8023"). InnerVolumeSpecName "kube-api-access-cvxk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:59:17.130794 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.130654    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fde313ca-17ff-426b-814f-f27b28fd8023-istio-token" pod "fde313ca-17ff-426b-814f-f27b28fd8023" (UID: "fde313ca-17ff-426b-814f-f27b28fd8023"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:59:17.130828 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.130815    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-podinfo" pod "2e4522dd-634e-4890-8ba5-67676e1575a9" (UID: "2e4522dd-634e-4890-8ba5-67676e1575a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:59:17.131847 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.131826    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-token" pod "2e4522dd-634e-4890-8ba5-67676e1575a9" (UID: "2e4522dd-634e-4890-8ba5-67676e1575a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:59:17.131847 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.131841    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e4522dd-634e-4890-8ba5-67676e1575a9-kube-api-access-9wwh6" pod "2e4522dd-634e-4890-8ba5-67676e1575a9" (UID: "2e4522dd-634e-4890-8ba5-67676e1575a9"). InnerVolumeSpecName "kube-api-access-9wwh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:59:17.228711 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.228675    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-envoy\") pod \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") "
Nov 04 01:59:17.228875 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.228756    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-data\") pod \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") "
Nov 04 01:59:17.228966 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.228887    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-kube-api-access-fqw8h\" (UniqueName: \"kubernetes.io/projected/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-kube-api-access-fqw8h\") pod \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") "
Nov 04 01:59:17.228966 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.228933    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-workload-certs\") pod \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") "
Nov 04 01:59:17.229078 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.228982    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-credential-socket\") pod \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") "
Nov 04 01:59:17.229078 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229032    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-podinfo\") pod \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") "
Nov 04 01:59:17.229078 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229059    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-token\" (UniqueName: \"kubernetes.io/projected/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-token\") pod \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") "
Nov 04 01:59:17.229229 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229078    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-data" pod "a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" (UID: "a7e923eb-3db7-4f1d-a9de-fa4a20620bfd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.229229 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229092    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istiod-ca-cert\") pod \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") "
Nov 04 01:59:17.229229 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229160    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-workload-socket\") pod \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\" (UID: \"a7e923eb-3db7-4f1d-a9de-fa4a20620bfd\") "
Nov 04 01:59:17.229415 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229393    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istiod-ca-cert" pod "a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" (UID: "a7e923eb-3db7-4f1d-a9de-fa4a20620bfd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:59:17.229726 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229682    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-workload-socket" pod "a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" (UID: "a7e923eb-3db7-4f1d-a9de-fa4a20620bfd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.229820 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229709    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.229820 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229747    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-workload-certs" pod "a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" (UID: "a7e923eb-3db7-4f1d-a9de-fa4a20620bfd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.229820 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229755    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e4522dd-634e-4890-8ba5-67676e1575a9-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.229820 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229793    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fde313ca-17ff-426b-814f-f27b28fd8023-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.229820 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229812    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.230020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229826    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.230020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229841    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvxk8\" (UniqueName: \"kubernetes.io/projected/fde313ca-17ff-426b-814f-f27b28fd8023-kube-api-access-cvxk8\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.230020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229856    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.230020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229869    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fde313ca-17ff-426b-814f-f27b28fd8023-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.230020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229883    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fde313ca-17ff-426b-814f-f27b28fd8023-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.230020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229897    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fde313ca-17ff-426b-814f-f27b28fd8023-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.230020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229910    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.230020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229927    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.230020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229941    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wwh6\" (UniqueName: \"kubernetes.io/projected/2e4522dd-634e-4890-8ba5-67676e1575a9-kube-api-access-9wwh6\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.230020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229956    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.230020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229968    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.230020 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.229981    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e4522dd-634e-4890-8ba5-67676e1575a9-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.230359 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.230070    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-credential-socket" pod "a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" (UID: "a7e923eb-3db7-4f1d-a9de-fa4a20620bfd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.231649 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.231620    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-envoy" pod "a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" (UID: "a7e923eb-3db7-4f1d-a9de-fa4a20620bfd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:17.231739 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.231672    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-podinfo" pod "a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" (UID: "a7e923eb-3db7-4f1d-a9de-fa4a20620bfd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:59:17.232024 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.232000    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-kube-api-access-fqw8h" pod "a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" (UID: "a7e923eb-3db7-4f1d-a9de-fa4a20620bfd"). InnerVolumeSpecName "kube-api-access-fqw8h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:59:17.232121 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.232079    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-token" pod "a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" (UID: "a7e923eb-3db7-4f1d-a9de-fa4a20620bfd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:59:17.331320 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.331261    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqw8h\" (UniqueName: \"kubernetes.io/projected/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-kube-api-access-fqw8h\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.331320 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.331322    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.331614 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.331338    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.331614 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.331351    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.331614 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.331366    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.331614 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.331378    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.331614 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.331390    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:17.555345 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.555254    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"]
Nov 04 01:59:17.560160 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.560133    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74"]
Nov 04 01:59:17.669541 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.669506    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"]
Nov 04 01:59:17.673205 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.673179    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q"]
Nov 04 01:59:17.782284 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.782245    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"]
Nov 04 01:59:17.786249 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.786225    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m"]
Nov 04 01:59:17.895193 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.895113    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"]
Nov 04 01:59:17.899095 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:17.899066    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5"]
Nov 04 01:59:18.141470 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:59:18.141407    6116 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-84cffd7d1248: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:59:18.141470 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:59:18.141438    6116 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k: secret "agc-cred-84cffd7d1248" not found
Nov 04 01:59:18.141946 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:59:18.141546    6116 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials podName:233cb7a1-2c92-4544-afe5-5aae953288e1 nodeName:}" failed. No retries permitted until 2026-11-04 02:01:20.141524637 +0000 UTC m=+1565.262802161 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials") pod "payload-processing-external-model-5497dccdcc-bbs4k" (UID: "233cb7a1-2c92-4544-afe5-5aae953288e1") : secret "agc-cred-84cffd7d1248" not found
Nov 04 01:59:18.333684 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.333650    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw"]
Nov 04 01:59:18.334046 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334034    6116 cpu_manager.go:414] "containerMap: removing container" podUID="33450851-a5a5-4567-8e86-b85e478d3bce" containerName="istio-proxy"
Nov 04 01:59:18.334084 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334048    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33450851-a5a5-4567-8e86-b85e478d3bce" containerName="istio-proxy"
Nov 04 01:59:18.334084 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334062    6116 cpu_manager.go:414] "containerMap: removing container" podUID="fde313ca-17ff-426b-814f-f27b28fd8023" containerName="istio-proxy"
Nov 04 01:59:18.334084 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334067    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fde313ca-17ff-426b-814f-f27b28fd8023" containerName="istio-proxy"
Nov 04 01:59:18.334084 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334081    6116 cpu_manager.go:414] "containerMap: removing container" podUID="a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334086    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334097    6116 cpu_manager.go:414] "containerMap: removing container" podUID="6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334102    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334112    6116 cpu_manager.go:414] "containerMap: removing container" podUID="30488fc4-bfaf-414c-9dcd-6e250398b8dc" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334116    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30488fc4-bfaf-414c-9dcd-6e250398b8dc" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334122    6116 cpu_manager.go:414] "containerMap: removing container" podUID="30488fc4-bfaf-414c-9dcd-6e250398b8dc" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334126    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30488fc4-bfaf-414c-9dcd-6e250398b8dc" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334141    6116 cpu_manager.go:414] "containerMap: removing container" podUID="dad0a1c4-88b2-40ee-9a4a-8997fc743448" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334145    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dad0a1c4-88b2-40ee-9a4a-8997fc743448" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334155    6116 cpu_manager.go:414] "containerMap: removing container" podUID="2e4522dd-634e-4890-8ba5-67676e1575a9" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334159    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e4522dd-634e-4890-8ba5-67676e1575a9" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334165    6116 cpu_manager.go:414] "containerMap: removing container" podUID="daa8ad32-cb35-48e2-a4b9-5dc397ce962b" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334170    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="daa8ad32-cb35-48e2-a4b9-5dc397ce962b" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334227    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e4522dd-634e-4890-8ba5-67676e1575a9" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334233    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="daa8ad32-cb35-48e2-a4b9-5dc397ce962b" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334239    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="33450851-a5a5-4567-8e86-b85e478d3bce" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334246    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="fde313ca-17ff-426b-814f-f27b28fd8023" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334253    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" containerName="istio-proxy"
Nov 04 01:59:18.334251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334259    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ce4bd82-51e8-4b93-a2fd-db2e721cbcf9" containerName="istio-proxy"
Nov 04 01:59:18.334763 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334264    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="30488fc4-bfaf-414c-9dcd-6e250398b8dc" containerName="istio-proxy"
Nov 04 01:59:18.334763 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334268    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="30488fc4-bfaf-414c-9dcd-6e250398b8dc" containerName="istio-proxy"
Nov 04 01:59:18.334763 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.334274    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="dad0a1c4-88b2-40ee-9a4a-8997fc743448" containerName="istio-proxy"
Nov 04 01:59:18.338741 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.338724    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw"
Nov 04 01:59:18.342887 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.342860    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9c6d88e0-1752-407e-8718-c8a64bce770a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7f4748e0-6rpdw\" (UID: \"9c6d88e0-1752-407e-8718-c8a64bce770a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw"
Nov 04 01:59:18.343009 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.342979    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6h26\" (UniqueName: \"kubernetes.io/projected/9c6d88e0-1752-407e-8718-c8a64bce770a-kube-api-access-d6h26\") pod \"maas-api-revoke-sub-7f4748e0-6rpdw\" (UID: \"9c6d88e0-1752-407e-8718-c8a64bce770a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw"
Nov 04 01:59:18.343052 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.343019    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9c6d88e0-1752-407e-8718-c8a64bce770a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7f4748e0-6rpdw\" (UID: \"9c6d88e0-1752-407e-8718-c8a64bce770a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw"
Nov 04 01:59:18.392049 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.392021    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:59:18.392251 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.392074    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:59:18.392430 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.392188    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 01:59:18.392556 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.392490    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bj8b6\""
Nov 04 01:59:18.394924 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.394901    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw"]
Nov 04 01:59:18.443506 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.443439    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6h26\" (UniqueName: \"kubernetes.io/projected/9c6d88e0-1752-407e-8718-c8a64bce770a-kube-api-access-d6h26\") pod \"maas-api-revoke-sub-7f4748e0-6rpdw\" (UID: \"9c6d88e0-1752-407e-8718-c8a64bce770a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw"
Nov 04 01:59:18.443506 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.443516    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9c6d88e0-1752-407e-8718-c8a64bce770a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7f4748e0-6rpdw\" (UID: \"9c6d88e0-1752-407e-8718-c8a64bce770a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw"
Nov 04 01:59:18.443718 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.443592    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9c6d88e0-1752-407e-8718-c8a64bce770a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7f4748e0-6rpdw\" (UID: \"9c6d88e0-1752-407e-8718-c8a64bce770a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw"
Nov 04 01:59:18.444204 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.444179    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9c6d88e0-1752-407e-8718-c8a64bce770a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7f4748e0-6rpdw\" (UID: \"9c6d88e0-1752-407e-8718-c8a64bce770a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw"
Nov 04 01:59:18.446125 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.446104    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9c6d88e0-1752-407e-8718-c8a64bce770a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7f4748e0-6rpdw\" (UID: \"9c6d88e0-1752-407e-8718-c8a64bce770a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw"
Nov 04 01:59:18.504049 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.504014    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6h26\" (UniqueName: \"kubernetes.io/projected/9c6d88e0-1752-407e-8718-c8a64bce770a-kube-api-access-d6h26\") pod \"maas-api-revoke-sub-7f4748e0-6rpdw\" (UID: \"9c6d88e0-1752-407e-8718-c8a64bce770a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw"
Nov 04 01:59:18.651641 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:18.651609    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw"
Nov 04 01:59:19.026409 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:59:19.026372    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c6d88e0_1752_407e_8718_c8a64bce770a.slice/crio-d33ab7818741f8720aef30dacbf6716948cb0c247f37bad8c958aec119d0d774 WatchSource:0}: Error finding container d33ab7818741f8720aef30dacbf6716948cb0c247f37bad8c958aec119d0d774: Status 404 returned error can't find the container with id d33ab7818741f8720aef30dacbf6716948cb0c247f37bad8c958aec119d0d774
Nov 04 01:59:19.028297 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:19.028260    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e4522dd-634e-4890-8ba5-67676e1575a9" path="/var/lib/kubelet/pods/2e4522dd-634e-4890-8ba5-67676e1575a9/volumes"
Nov 04 01:59:19.028867 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:19.028847    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7e923eb-3db7-4f1d-a9de-fa4a20620bfd" path="/var/lib/kubelet/pods/a7e923eb-3db7-4f1d-a9de-fa4a20620bfd/volumes"
Nov 04 01:59:19.029433 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:19.029409    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dad0a1c4-88b2-40ee-9a4a-8997fc743448" path="/var/lib/kubelet/pods/dad0a1c4-88b2-40ee-9a4a-8997fc743448/volumes"
Nov 04 01:59:19.030390 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:19.030367    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fde313ca-17ff-426b-814f-f27b28fd8023" path="/var/lib/kubelet/pods/fde313ca-17ff-426b-814f-f27b28fd8023/volumes"
Nov 04 01:59:19.030787 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:19.030768    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw"]
Nov 04 01:59:19.109071 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:19.109040    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw" event={"ID":"9c6d88e0-1752-407e-8718-c8a64bce770a","Type":"ContainerStarted","Data":"d33ab7818741f8720aef30dacbf6716948cb0c247f37bad8c958aec119d0d774"}
Nov 04 01:59:20.116005 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:20.115970    6116 generic.go:356] "Generic (PLEG): container finished" podID="9c6d88e0-1752-407e-8718-c8a64bce770a" containerID="fd72cfb5eae4bd715a89df9e9ff5769c9a8c344565654f21c4225f5e9175d4d3" exitCode=0
Nov 04 01:59:20.116409 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:20.116068    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw" event={"ID":"9c6d88e0-1752-407e-8718-c8a64bce770a","Type":"ContainerDied","Data":"fd72cfb5eae4bd715a89df9e9ff5769c9a8c344565654f21c4225f5e9175d4d3"}
Nov 04 01:59:21.278579 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:21.278552    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw"
Nov 04 01:59:21.374260 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:21.374181    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9c6d88e0-1752-407e-8718-c8a64bce770a-kube-api-access-d6h26\" (UniqueName: \"kubernetes.io/projected/9c6d88e0-1752-407e-8718-c8a64bce770a-kube-api-access-d6h26\") pod \"9c6d88e0-1752-407e-8718-c8a64bce770a\" (UID: \"9c6d88e0-1752-407e-8718-c8a64bce770a\") "
Nov 04 01:59:21.374260 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:21.374222    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9c6d88e0-1752-407e-8718-c8a64bce770a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9c6d88e0-1752-407e-8718-c8a64bce770a-maas-api-xks-ca\") pod \"9c6d88e0-1752-407e-8718-c8a64bce770a\" (UID: \"9c6d88e0-1752-407e-8718-c8a64bce770a\") "
Nov 04 01:59:21.374552 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:21.374323    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9c6d88e0-1752-407e-8718-c8a64bce770a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9c6d88e0-1752-407e-8718-c8a64bce770a-maas-api-openshift-ca\") pod \"9c6d88e0-1752-407e-8718-c8a64bce770a\" (UID: \"9c6d88e0-1752-407e-8718-c8a64bce770a\") "
Nov 04 01:59:21.374773 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:21.374744    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c6d88e0-1752-407e-8718-c8a64bce770a-maas-api-openshift-ca" pod "9c6d88e0-1752-407e-8718-c8a64bce770a" (UID: "9c6d88e0-1752-407e-8718-c8a64bce770a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:59:21.376537 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:21.376510    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c6d88e0-1752-407e-8718-c8a64bce770a-maas-api-xks-ca" pod "9c6d88e0-1752-407e-8718-c8a64bce770a" (UID: "9c6d88e0-1752-407e-8718-c8a64bce770a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:59:21.376624 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:21.376573    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c6d88e0-1752-407e-8718-c8a64bce770a-kube-api-access-d6h26" pod "9c6d88e0-1752-407e-8718-c8a64bce770a" (UID: "9c6d88e0-1752-407e-8718-c8a64bce770a"). InnerVolumeSpecName "kube-api-access-d6h26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:59:21.476027 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:21.475988    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9c6d88e0-1752-407e-8718-c8a64bce770a-maas-api-xks-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:21.476027 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:21.476027    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9c6d88e0-1752-407e-8718-c8a64bce770a-maas-api-openshift-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:21.476231 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:21.476042    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6h26\" (UniqueName: \"kubernetes.io/projected/9c6d88e0-1752-407e-8718-c8a64bce770a-kube-api-access-d6h26\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:22.129598 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:22.129571    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw"
Nov 04 01:59:22.129856 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:22.129567    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f4748e0-6rpdw" event={"ID":"9c6d88e0-1752-407e-8718-c8a64bce770a","Type":"ContainerDied","Data":"d33ab7818741f8720aef30dacbf6716948cb0c247f37bad8c958aec119d0d774"}
Nov 04 01:59:22.129856 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:22.129687    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d33ab7818741f8720aef30dacbf6716948cb0c247f37bad8c958aec119d0d774"
Nov 04 01:59:31.760885 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:31.760797    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"]
Nov 04 01:59:34.667304 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.667266    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc"]
Nov 04 01:59:34.667730 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.667672    6116 cpu_manager.go:414] "containerMap: removing container" podUID="9c6d88e0-1752-407e-8718-c8a64bce770a" containerName="revoke-keys"
Nov 04 01:59:34.667730 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.667683    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c6d88e0-1752-407e-8718-c8a64bce770a" containerName="revoke-keys"
Nov 04 01:59:34.667804 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.667767    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c6d88e0-1752-407e-8718-c8a64bce770a" containerName="revoke-keys"
Nov 04 01:59:34.670879 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.670859    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc"
Nov 04 01:59:34.775556 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.775512    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 01:59:34.775556 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.775533    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bj8b6\""
Nov 04 01:59:34.775556 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.775534    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:59:34.775793 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.775563    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:59:34.777848 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.777828    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc"]
Nov 04 01:59:34.825456 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.825423    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1f4bd2f5-7027-4f62-8123-b222884e9bbe-maas-api-xks-ca\") pod \"maas-api-revoke-sub-57ae934e-prsjc\" (UID: \"1f4bd2f5-7027-4f62-8123-b222884e9bbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc"
Nov 04 01:59:34.825647 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.825623    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlsbj\" (UniqueName: \"kubernetes.io/projected/1f4bd2f5-7027-4f62-8123-b222884e9bbe-kube-api-access-qlsbj\") pod \"maas-api-revoke-sub-57ae934e-prsjc\" (UID: \"1f4bd2f5-7027-4f62-8123-b222884e9bbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc"
Nov 04 01:59:34.825717 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.825675    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1f4bd2f5-7027-4f62-8123-b222884e9bbe-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-57ae934e-prsjc\" (UID: \"1f4bd2f5-7027-4f62-8123-b222884e9bbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc"
Nov 04 01:59:34.926564 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.926471    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlsbj\" (UniqueName: \"kubernetes.io/projected/1f4bd2f5-7027-4f62-8123-b222884e9bbe-kube-api-access-qlsbj\") pod \"maas-api-revoke-sub-57ae934e-prsjc\" (UID: \"1f4bd2f5-7027-4f62-8123-b222884e9bbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc"
Nov 04 01:59:34.926564 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.926519    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1f4bd2f5-7027-4f62-8123-b222884e9bbe-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-57ae934e-prsjc\" (UID: \"1f4bd2f5-7027-4f62-8123-b222884e9bbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc"
Nov 04 01:59:34.926796 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.926601    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1f4bd2f5-7027-4f62-8123-b222884e9bbe-maas-api-xks-ca\") pod \"maas-api-revoke-sub-57ae934e-prsjc\" (UID: \"1f4bd2f5-7027-4f62-8123-b222884e9bbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc"
Nov 04 01:59:34.927197 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.927174    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1f4bd2f5-7027-4f62-8123-b222884e9bbe-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-57ae934e-prsjc\" (UID: \"1f4bd2f5-7027-4f62-8123-b222884e9bbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc"
Nov 04 01:59:34.929009 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.928987    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1f4bd2f5-7027-4f62-8123-b222884e9bbe-maas-api-xks-ca\") pod \"maas-api-revoke-sub-57ae934e-prsjc\" (UID: \"1f4bd2f5-7027-4f62-8123-b222884e9bbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc"
Nov 04 01:59:34.987241 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:34.987205    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlsbj\" (UniqueName: \"kubernetes.io/projected/1f4bd2f5-7027-4f62-8123-b222884e9bbe-kube-api-access-qlsbj\") pod \"maas-api-revoke-sub-57ae934e-prsjc\" (UID: \"1f4bd2f5-7027-4f62-8123-b222884e9bbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc"
Nov 04 01:59:35.283346 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:35.283267    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc"
Nov 04 01:59:35.655298 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:35.655263    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc"]
Nov 04 01:59:35.656340 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:59:35.656315    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f4bd2f5_7027_4f62_8123_b222884e9bbe.slice/crio-4309ab5d0570dbda5422c4a477ad4b313b1f013869c9984f779a705c691b0bb7 WatchSource:0}: Error finding container 4309ab5d0570dbda5422c4a477ad4b313b1f013869c9984f779a705c691b0bb7: Status 404 returned error can't find the container with id 4309ab5d0570dbda5422c4a477ad4b313b1f013869c9984f779a705c691b0bb7
Nov 04 01:59:36.146955 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:36.146919    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:59:36.209772 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:36.209738    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc" event={"ID":"1f4bd2f5-7027-4f62-8123-b222884e9bbe","Type":"ContainerStarted","Data":"aa1f6576c624012f5ce48f65b6df6bdd8069773c095df5e29199d76fb2241445"}
Nov 04 01:59:36.209938 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:36.209789    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc" event={"ID":"1f4bd2f5-7027-4f62-8123-b222884e9bbe","Type":"ContainerStarted","Data":"4309ab5d0570dbda5422c4a477ad4b313b1f013869c9984f779a705c691b0bb7"}
Nov 04 01:59:36.327493 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:36.327375    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc" podStartSLOduration=2.327360465 podStartE2EDuration="2.327360465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:59:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:59:36.324233505 +0000 UTC m=+1461.445511031" watchObservedRunningTime="2026-11-04 01:59:36.327360465 +0000 UTC m=+1461.448638023"
Nov 04 01:59:36.761912 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:36.761862    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k" podUID="233cb7a1-2c92-4544-afe5-5aae953288e1" containerName="payload-processing" containerID="cri-o://cc683b087ce900646610d30fc54e45c04ca554222c2446f27273a4f5b3fc93f9" gracePeriod=25
Nov 04 01:59:37.012255 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.012199    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"
Nov 04 01:59:37.148715 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.148676    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-kube-api-access-c56hd\" (UniqueName: \"kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-kube-api-access-c56hd\") pod \"233cb7a1-2c92-4544-afe5-5aae953288e1\" (UID: \"233cb7a1-2c92-4544-afe5-5aae953288e1\") "
Nov 04 01:59:37.148715 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.148707    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/233cb7a1-2c92-4544-afe5-5aae953288e1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/233cb7a1-2c92-4544-afe5-5aae953288e1-plugins-config-volume\") pod \"233cb7a1-2c92-4544-afe5-5aae953288e1\" (UID: \"233cb7a1-2c92-4544-afe5-5aae953288e1\") "
Nov 04 01:59:37.149276 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.148727    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/233cb7a1-2c92-4544-afe5-5aae953288e1-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/233cb7a1-2c92-4544-afe5-5aae953288e1-routing-overlay-volume\") pod \"233cb7a1-2c92-4544-afe5-5aae953288e1\" (UID: \"233cb7a1-2c92-4544-afe5-5aae953288e1\") "
Nov 04 01:59:37.149276 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.148755    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials\" (UniqueName: \"kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials\") pod \"233cb7a1-2c92-4544-afe5-5aae953288e1\" (UID: \"233cb7a1-2c92-4544-afe5-5aae953288e1\") "
Nov 04 01:59:37.149276 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.149110    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/233cb7a1-2c92-4544-afe5-5aae953288e1-routing-overlay-volume" pod "233cb7a1-2c92-4544-afe5-5aae953288e1" (UID: "233cb7a1-2c92-4544-afe5-5aae953288e1"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:59:37.149276 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.149166    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/233cb7a1-2c92-4544-afe5-5aae953288e1-plugins-config-volume" pod "233cb7a1-2c92-4544-afe5-5aae953288e1" (UID: "233cb7a1-2c92-4544-afe5-5aae953288e1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:59:37.150980 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.150955    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-kube-api-access-c56hd" pod "233cb7a1-2c92-4544-afe5-5aae953288e1" (UID: "233cb7a1-2c92-4544-afe5-5aae953288e1"). InnerVolumeSpecName "kube-api-access-c56hd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:59:37.151114 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.150993    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials" pod "233cb7a1-2c92-4544-afe5-5aae953288e1" (UID: "233cb7a1-2c92-4544-afe5-5aae953288e1"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:59:37.216263 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.216232    6116 generic.go:356] "Generic (PLEG): container finished" podID="1f4bd2f5-7027-4f62-8123-b222884e9bbe" containerID="aa1f6576c624012f5ce48f65b6df6bdd8069773c095df5e29199d76fb2241445" exitCode=0
Nov 04 01:59:37.216417 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.216328    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc" event={"ID":"1f4bd2f5-7027-4f62-8123-b222884e9bbe","Type":"ContainerDied","Data":"aa1f6576c624012f5ce48f65b6df6bdd8069773c095df5e29199d76fb2241445"}
Nov 04 01:59:37.217641 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.217620    6116 generic.go:356] "Generic (PLEG): container finished" podID="233cb7a1-2c92-4544-afe5-5aae953288e1" containerID="cc683b087ce900646610d30fc54e45c04ca554222c2446f27273a4f5b3fc93f9" exitCode=0
Nov 04 01:59:37.217744 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.217660    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"
Nov 04 01:59:37.217744 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.217671    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k" event={"ID":"233cb7a1-2c92-4544-afe5-5aae953288e1","Type":"ContainerDied","Data":"cc683b087ce900646610d30fc54e45c04ca554222c2446f27273a4f5b3fc93f9"}
Nov 04 01:59:37.217744 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.217702    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k" event={"ID":"233cb7a1-2c92-4544-afe5-5aae953288e1","Type":"ContainerDied","Data":"b3746d9fade88be3205242fb4d5b3bbdd2a3f7a10c9063f654dce3f05f9a2c9b"}
Nov 04 01:59:37.217744 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.217724    6116 scope.go:122] "RemoveContainer" containerID="cc683b087ce900646610d30fc54e45c04ca554222c2446f27273a4f5b3fc93f9"
Nov 04 01:59:37.227121 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.227105    6116 scope.go:122] "RemoveContainer" containerID="cc683b087ce900646610d30fc54e45c04ca554222c2446f27273a4f5b3fc93f9"
Nov 04 01:59:37.227405 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:59:37.227388    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc683b087ce900646610d30fc54e45c04ca554222c2446f27273a4f5b3fc93f9\": container with ID starting with cc683b087ce900646610d30fc54e45c04ca554222c2446f27273a4f5b3fc93f9 not found: ID does not exist" containerID="cc683b087ce900646610d30fc54e45c04ca554222c2446f27273a4f5b3fc93f9"
Nov 04 01:59:37.227464 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.227412    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc683b087ce900646610d30fc54e45c04ca554222c2446f27273a4f5b3fc93f9"} err="failed to get container status \"cc683b087ce900646610d30fc54e45c04ca554222c2446f27273a4f5b3fc93f9\": rpc error: code = NotFound desc = could not find container \"cc683b087ce900646610d30fc54e45c04ca554222c2446f27273a4f5b3fc93f9\": container with ID starting with cc683b087ce900646610d30fc54e45c04ca554222c2446f27273a4f5b3fc93f9 not found: ID does not exist"
Nov 04 01:59:37.249989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.249961    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c56hd\" (UniqueName: \"kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-kube-api-access-c56hd\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:37.249989 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.249985    6116 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/233cb7a1-2c92-4544-afe5-5aae953288e1-plugins-config-volume\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:37.250160 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.249995    6116 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/233cb7a1-2c92-4544-afe5-5aae953288e1-routing-overlay-volume\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:37.250160 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.250005    6116 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/233cb7a1-2c92-4544-afe5-5aae953288e1-provider-credentials\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:37.442349 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.442317    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"]
Nov 04 01:59:37.448591 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:37.448566    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k"]
Nov 04 01:59:38.373682 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:38.373656    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc"
Nov 04 01:59:38.462117 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:38.462087    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f4bd2f5-7027-4f62-8123-b222884e9bbe-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1f4bd2f5-7027-4f62-8123-b222884e9bbe-maas-api-openshift-ca\") pod \"1f4bd2f5-7027-4f62-8123-b222884e9bbe\" (UID: \"1f4bd2f5-7027-4f62-8123-b222884e9bbe\") "
Nov 04 01:59:38.462284 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:38.462133    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f4bd2f5-7027-4f62-8123-b222884e9bbe-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1f4bd2f5-7027-4f62-8123-b222884e9bbe-maas-api-xks-ca\") pod \"1f4bd2f5-7027-4f62-8123-b222884e9bbe\" (UID: \"1f4bd2f5-7027-4f62-8123-b222884e9bbe\") "
Nov 04 01:59:38.462284 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:38.462180    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f4bd2f5-7027-4f62-8123-b222884e9bbe-kube-api-access-qlsbj\" (UniqueName: \"kubernetes.io/projected/1f4bd2f5-7027-4f62-8123-b222884e9bbe-kube-api-access-qlsbj\") pod \"1f4bd2f5-7027-4f62-8123-b222884e9bbe\" (UID: \"1f4bd2f5-7027-4f62-8123-b222884e9bbe\") "
Nov 04 01:59:38.462529 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:38.462509    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f4bd2f5-7027-4f62-8123-b222884e9bbe-maas-api-openshift-ca" pod "1f4bd2f5-7027-4f62-8123-b222884e9bbe" (UID: "1f4bd2f5-7027-4f62-8123-b222884e9bbe"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:59:38.464371 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:38.464345    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f4bd2f5-7027-4f62-8123-b222884e9bbe-kube-api-access-qlsbj" pod "1f4bd2f5-7027-4f62-8123-b222884e9bbe" (UID: "1f4bd2f5-7027-4f62-8123-b222884e9bbe"). InnerVolumeSpecName "kube-api-access-qlsbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:59:38.464371 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:38.464362    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f4bd2f5-7027-4f62-8123-b222884e9bbe-maas-api-xks-ca" pod "1f4bd2f5-7027-4f62-8123-b222884e9bbe" (UID: "1f4bd2f5-7027-4f62-8123-b222884e9bbe"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:59:38.563986 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:38.563888    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1f4bd2f5-7027-4f62-8123-b222884e9bbe-maas-api-openshift-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:38.563986 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:38.563916    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1f4bd2f5-7027-4f62-8123-b222884e9bbe-maas-api-xks-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:38.563986 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:38.563927    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlsbj\" (UniqueName: \"kubernetes.io/projected/1f4bd2f5-7027-4f62-8123-b222884e9bbe-kube-api-access-qlsbj\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:38.679551 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:38.679515    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"]
Nov 04 01:59:39.026378 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:39.026345    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="233cb7a1-2c92-4544-afe5-5aae953288e1" path="/var/lib/kubelet/pods/233cb7a1-2c92-4544-afe5-5aae953288e1/volumes"
Nov 04 01:59:39.237055 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:39.237030    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc"
Nov 04 01:59:39.237227 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:39.237062    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57ae934e-prsjc" event={"ID":"1f4bd2f5-7027-4f62-8123-b222884e9bbe","Type":"ContainerDied","Data":"4309ab5d0570dbda5422c4a477ad4b313b1f013869c9984f779a705c691b0bb7"}
Nov 04 01:59:39.237227 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:39.237113    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4309ab5d0570dbda5422c4a477ad4b313b1f013869c9984f779a705c691b0bb7"
Nov 04 01:59:41.992058 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:59:41.992025    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 01:59:51.753273 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.753238    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms"]
Nov 04 01:59:51.753853 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.753832    6116 cpu_manager.go:414] "containerMap: removing container" podUID="1f4bd2f5-7027-4f62-8123-b222884e9bbe" containerName="revoke-keys"
Nov 04 01:59:51.753937 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.753856    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f4bd2f5-7027-4f62-8123-b222884e9bbe" containerName="revoke-keys"
Nov 04 01:59:51.753937 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.753889    6116 cpu_manager.go:414] "containerMap: removing container" podUID="233cb7a1-2c92-4544-afe5-5aae953288e1" containerName="payload-processing"
Nov 04 01:59:51.753937 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.753896    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="233cb7a1-2c92-4544-afe5-5aae953288e1" containerName="payload-processing"
Nov 04 01:59:51.754078 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.754013    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="233cb7a1-2c92-4544-afe5-5aae953288e1" containerName="payload-processing"
Nov 04 01:59:51.754078 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.754031    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f4bd2f5-7027-4f62-8123-b222884e9bbe" containerName="revoke-keys"
Nov 04 01:59:51.758376 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.758356    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms"
Nov 04 01:59:51.789993 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.789957    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtsnk\" (UniqueName: \"kubernetes.io/projected/929d4558-6de3-4bf9-894a-169498b5ed7b-kube-api-access-rtsnk\") pod \"maas-api-revoke-sub-8019849f-sgjms\" (UID: \"929d4558-6de3-4bf9-894a-169498b5ed7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms"
Nov 04 01:59:51.790149 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.790003    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/929d4558-6de3-4bf9-894a-169498b5ed7b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8019849f-sgjms\" (UID: \"929d4558-6de3-4bf9-894a-169498b5ed7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms"
Nov 04 01:59:51.790149 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.790127    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/929d4558-6de3-4bf9-894a-169498b5ed7b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8019849f-sgjms\" (UID: \"929d4558-6de3-4bf9-894a-169498b5ed7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms"
Nov 04 01:59:51.862609 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.862569    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 01:59:51.862797 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.862717    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bj8b6\""
Nov 04 01:59:51.862797 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.862757    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:59:51.862913 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.862827    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:59:51.865443 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.865419    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms"]
Nov 04 01:59:51.891222 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.891199    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/929d4558-6de3-4bf9-894a-169498b5ed7b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8019849f-sgjms\" (UID: \"929d4558-6de3-4bf9-894a-169498b5ed7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms"
Nov 04 01:59:51.891336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.891268    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtsnk\" (UniqueName: \"kubernetes.io/projected/929d4558-6de3-4bf9-894a-169498b5ed7b-kube-api-access-rtsnk\") pod \"maas-api-revoke-sub-8019849f-sgjms\" (UID: \"929d4558-6de3-4bf9-894a-169498b5ed7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms"
Nov 04 01:59:51.891336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.891296    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/929d4558-6de3-4bf9-894a-169498b5ed7b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8019849f-sgjms\" (UID: \"929d4558-6de3-4bf9-894a-169498b5ed7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms"
Nov 04 01:59:51.891890 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.891871    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/929d4558-6de3-4bf9-894a-169498b5ed7b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8019849f-sgjms\" (UID: \"929d4558-6de3-4bf9-894a-169498b5ed7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms"
Nov 04 01:59:51.893785 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.893764    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/929d4558-6de3-4bf9-894a-169498b5ed7b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8019849f-sgjms\" (UID: \"929d4558-6de3-4bf9-894a-169498b5ed7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms"
Nov 04 01:59:51.952287 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.952253    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtsnk\" (UniqueName: \"kubernetes.io/projected/929d4558-6de3-4bf9-894a-169498b5ed7b-kube-api-access-rtsnk\") pod \"maas-api-revoke-sub-8019849f-sgjms\" (UID: \"929d4558-6de3-4bf9-894a-169498b5ed7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms"
Nov 04 01:59:51.966399 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:51.966365    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh"]
Nov 04 01:59:52.070634 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:52.070555    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms"
Nov 04 01:59:52.074482 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:52.074420    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt"]
Nov 04 01:59:52.442993 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:52.442966    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms"]
Nov 04 01:59:52.444139 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:59:52.444114    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod929d4558_6de3_4bf9_894a_169498b5ed7b.slice/crio-1317d085d2b014fdbff9a420f3f5be2fbcaaed0440b223d3e4c2ff4f9d92d5cb WatchSource:0}: Error finding container 1317d085d2b014fdbff9a420f3f5be2fbcaaed0440b223d3e4c2ff4f9d92d5cb: Status 404 returned error can't find the container with id 1317d085d2b014fdbff9a420f3f5be2fbcaaed0440b223d3e4c2ff4f9d92d5cb
Nov 04 01:59:53.326240 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:53.326211    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms" event={"ID":"929d4558-6de3-4bf9-894a-169498b5ed7b","Type":"ContainerStarted","Data":"c18a70587948a3bff918976338b80c4e987aa31c54cee0d92c37842c2c3e2c82"}
Nov 04 01:59:53.326651 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:53.326256    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms" event={"ID":"929d4558-6de3-4bf9-894a-169498b5ed7b","Type":"ContainerStarted","Data":"1317d085d2b014fdbff9a420f3f5be2fbcaaed0440b223d3e4c2ff4f9d92d5cb"}
Nov 04 01:59:53.442121 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:53.442052    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms" podStartSLOduration=2.442037499 podStartE2EDuration="2.442037499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:59:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:59:53.441724886 +0000 UTC m=+1478.563002411" watchObservedRunningTime="2026-11-04 01:59:53.442037499 +0000 UTC m=+1478.563315026"
Nov 04 01:59:53.699440 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:53.699397    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj" podUID="a1501413-7507-4f1d-ae16-de82aed103ea" containerName="main" containerID="cri-o://fafa4c495d103497209c7a0f4bce8f4d17dd6db09e34118228c8ab04e95ffe52" gracePeriod=45
Nov 04 01:59:53.835372 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:53.835335    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj" podUID="a1501413-7507-4f1d-ae16-de82aed103ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": dial tcp 10.134.0.58:8000: connect: connection refused"
Nov 04 01:59:53.948952 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:53.948927    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:59:54.014736 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.014660    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a1501413-7507-4f1d-ae16-de82aed103ea-tls-certs\" (UniqueName: \"kubernetes.io/secret/a1501413-7507-4f1d-ae16-de82aed103ea-tls-certs\") pod \"a1501413-7507-4f1d-ae16-de82aed103ea\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") "
Nov 04 01:59:54.014736 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.014710    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-dshm\") pod \"a1501413-7507-4f1d-ae16-de82aed103ea\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") "
Nov 04 01:59:54.014736 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.014727    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-model-cache\") pod \"a1501413-7507-4f1d-ae16-de82aed103ea\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") "
Nov 04 01:59:54.015018 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.014755    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-tmp-dir\") pod \"a1501413-7507-4f1d-ae16-de82aed103ea\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") "
Nov 04 01:59:54.015018 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.014856    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1501413-7507-4f1d-ae16-de82aed103ea-kube-api-access-zb2sf\" (UniqueName: \"kubernetes.io/projected/a1501413-7507-4f1d-ae16-de82aed103ea-kube-api-access-zb2sf\") pod \"a1501413-7507-4f1d-ae16-de82aed103ea\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") "
Nov 04 01:59:54.015018 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.014888    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-home\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-home\") pod \"a1501413-7507-4f1d-ae16-de82aed103ea\" (UID: \"a1501413-7507-4f1d-ae16-de82aed103ea\") "
Nov 04 01:59:54.015176 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.015024    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-model-cache" pod "a1501413-7507-4f1d-ae16-de82aed103ea" (UID: "a1501413-7507-4f1d-ae16-de82aed103ea"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:54.015176 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.015098    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-tmp-dir" pod "a1501413-7507-4f1d-ae16-de82aed103ea" (UID: "a1501413-7507-4f1d-ae16-de82aed103ea"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:54.015176 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.015170    6116 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-model-cache\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:54.015323 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.015175    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-home" pod "a1501413-7507-4f1d-ae16-de82aed103ea" (UID: "a1501413-7507-4f1d-ae16-de82aed103ea"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:54.017038 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.017008    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1501413-7507-4f1d-ae16-de82aed103ea-tls-certs" pod "a1501413-7507-4f1d-ae16-de82aed103ea" (UID: "a1501413-7507-4f1d-ae16-de82aed103ea"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:59:54.017038 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.017016    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-dshm" pod "a1501413-7507-4f1d-ae16-de82aed103ea" (UID: "a1501413-7507-4f1d-ae16-de82aed103ea"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:59:54.017189 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.017065    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1501413-7507-4f1d-ae16-de82aed103ea-kube-api-access-zb2sf" pod "a1501413-7507-4f1d-ae16-de82aed103ea" (UID: "a1501413-7507-4f1d-ae16-de82aed103ea"). InnerVolumeSpecName "kube-api-access-zb2sf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:59:54.116669 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.116635    6116 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1501413-7507-4f1d-ae16-de82aed103ea-tls-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:54.116669 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.116661    6116 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-dshm\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:54.116669 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.116670    6116 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-tmp-dir\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:54.116669 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.116679    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zb2sf\" (UniqueName: \"kubernetes.io/projected/a1501413-7507-4f1d-ae16-de82aed103ea-kube-api-access-zb2sf\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:54.117043 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.116687    6116 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1501413-7507-4f1d-ae16-de82aed103ea-home\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:54.334213 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.334115    6116 generic.go:356] "Generic (PLEG): container finished" podID="a1501413-7507-4f1d-ae16-de82aed103ea" containerID="fafa4c495d103497209c7a0f4bce8f4d17dd6db09e34118228c8ab04e95ffe52" exitCode=0
Nov 04 01:59:54.334213 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.334157    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"
Nov 04 01:59:54.334733 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.334210    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj" event={"ID":"a1501413-7507-4f1d-ae16-de82aed103ea","Type":"ContainerDied","Data":"fafa4c495d103497209c7a0f4bce8f4d17dd6db09e34118228c8ab04e95ffe52"}
Nov 04 01:59:54.334733 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.334248    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj" event={"ID":"a1501413-7507-4f1d-ae16-de82aed103ea","Type":"ContainerDied","Data":"49a306715af304c202b1a6de5b150c7eb5b3f477802ab8b35043c06c0b5a5873"}
Nov 04 01:59:54.334733 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.334267    6116 scope.go:122] "RemoveContainer" containerID="fafa4c495d103497209c7a0f4bce8f4d17dd6db09e34118228c8ab04e95ffe52"
Nov 04 01:59:54.336911 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.336888    6116 generic.go:356] "Generic (PLEG): container finished" podID="929d4558-6de3-4bf9-894a-169498b5ed7b" containerID="c18a70587948a3bff918976338b80c4e987aa31c54cee0d92c37842c2c3e2c82" exitCode=0
Nov 04 01:59:54.337038 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.336994    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms" event={"ID":"929d4558-6de3-4bf9-894a-169498b5ed7b","Type":"ContainerDied","Data":"c18a70587948a3bff918976338b80c4e987aa31c54cee0d92c37842c2c3e2c82"}
Nov 04 01:59:54.345702 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.345686    6116 scope.go:122] "RemoveContainer" containerID="fafa4c495d103497209c7a0f4bce8f4d17dd6db09e34118228c8ab04e95ffe52"
Nov 04 01:59:54.345989 ip-10-0-129-138 kubenswrapper[6116]: E1104 01:59:54.345968    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fafa4c495d103497209c7a0f4bce8f4d17dd6db09e34118228c8ab04e95ffe52\": container with ID starting with fafa4c495d103497209c7a0f4bce8f4d17dd6db09e34118228c8ab04e95ffe52 not found: ID does not exist" containerID="fafa4c495d103497209c7a0f4bce8f4d17dd6db09e34118228c8ab04e95ffe52"
Nov 04 01:59:54.346053 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.346002    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fafa4c495d103497209c7a0f4bce8f4d17dd6db09e34118228c8ab04e95ffe52"} err="failed to get container status \"fafa4c495d103497209c7a0f4bce8f4d17dd6db09e34118228c8ab04e95ffe52\": rpc error: code = NotFound desc = could not find container \"fafa4c495d103497209c7a0f4bce8f4d17dd6db09e34118228c8ab04e95ffe52\": container with ID starting with fafa4c495d103497209c7a0f4bce8f4d17dd6db09e34118228c8ab04e95ffe52 not found: ID does not exist"
Nov 04 01:59:54.621959 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.621875    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"]
Nov 04 01:59:54.639165 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:54.639135    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-7af38e/premium-w0-7af38e-kserve-75685b786b-xmcwj"]
Nov 04 01:59:55.026675 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:55.026645    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1501413-7507-4f1d-ae16-de82aed103ea" path="/var/lib/kubelet/pods/a1501413-7507-4f1d-ae16-de82aed103ea/volumes"
Nov 04 01:59:55.476465 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:55.476431    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms"
Nov 04 01:59:55.529189 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:55.529157    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/929d4558-6de3-4bf9-894a-169498b5ed7b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/929d4558-6de3-4bf9-894a-169498b5ed7b-maas-api-xks-ca\") pod \"929d4558-6de3-4bf9-894a-169498b5ed7b\" (UID: \"929d4558-6de3-4bf9-894a-169498b5ed7b\") "
Nov 04 01:59:55.529189 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:55.529191    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/929d4558-6de3-4bf9-894a-169498b5ed7b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/929d4558-6de3-4bf9-894a-169498b5ed7b-maas-api-openshift-ca\") pod \"929d4558-6de3-4bf9-894a-169498b5ed7b\" (UID: \"929d4558-6de3-4bf9-894a-169498b5ed7b\") "
Nov 04 01:59:55.529439 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:55.529252    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/929d4558-6de3-4bf9-894a-169498b5ed7b-kube-api-access-rtsnk\" (UniqueName: \"kubernetes.io/projected/929d4558-6de3-4bf9-894a-169498b5ed7b-kube-api-access-rtsnk\") pod \"929d4558-6de3-4bf9-894a-169498b5ed7b\" (UID: \"929d4558-6de3-4bf9-894a-169498b5ed7b\") "
Nov 04 01:59:55.529578 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:55.529556    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/929d4558-6de3-4bf9-894a-169498b5ed7b-maas-api-openshift-ca" pod "929d4558-6de3-4bf9-894a-169498b5ed7b" (UID: "929d4558-6de3-4bf9-894a-169498b5ed7b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:59:55.531429 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:55.531410    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/929d4558-6de3-4bf9-894a-169498b5ed7b-kube-api-access-rtsnk" pod "929d4558-6de3-4bf9-894a-169498b5ed7b" (UID: "929d4558-6de3-4bf9-894a-169498b5ed7b"). InnerVolumeSpecName "kube-api-access-rtsnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:59:55.531429 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:55.531422    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/929d4558-6de3-4bf9-894a-169498b5ed7b-maas-api-xks-ca" pod "929d4558-6de3-4bf9-894a-169498b5ed7b" (UID: "929d4558-6de3-4bf9-894a-169498b5ed7b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:59:55.630422 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:55.630339    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/929d4558-6de3-4bf9-894a-169498b5ed7b-maas-api-xks-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:55.630422 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:55.630367    6116 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/929d4558-6de3-4bf9-894a-169498b5ed7b-maas-api-openshift-ca\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:55.630422 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:55.630380    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtsnk\" (UniqueName: \"kubernetes.io/projected/929d4558-6de3-4bf9-894a-169498b5ed7b-kube-api-access-rtsnk\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 01:59:56.350473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:56.350421    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms" event={"ID":"929d4558-6de3-4bf9-894a-169498b5ed7b","Type":"ContainerDied","Data":"1317d085d2b014fdbff9a420f3f5be2fbcaaed0440b223d3e4c2ff4f9d92d5cb"}
Nov 04 01:59:56.350642 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:56.350486    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8019849f-sgjms"
Nov 04 01:59:56.350642 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:56.350496    6116 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1317d085d2b014fdbff9a420f3f5be2fbcaaed0440b223d3e4c2ff4f9d92d5cb"
Nov 04 01:59:56.967533 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:56.967492    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh" podUID="aaa5bb34-fdba-497a-b58b-9ca549fa3d18" containerName="payload-processing" containerID="cri-o://f7bce7de817c496175624931d6f5883e8d21d0290825947f97952edc8ab38da4" gracePeriod=25
Nov 04 01:59:56.977170 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:56.977143    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"]
Nov 04 01:59:56.977580 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:56.977566    6116 cpu_manager.go:414] "containerMap: removing container" podUID="a1501413-7507-4f1d-ae16-de82aed103ea" containerName="main"
Nov 04 01:59:56.977626 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:56.977582    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1501413-7507-4f1d-ae16-de82aed103ea" containerName="main"
Nov 04 01:59:56.977626 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:56.977601    6116 cpu_manager.go:414] "containerMap: removing container" podUID="929d4558-6de3-4bf9-894a-169498b5ed7b" containerName="revoke-keys"
Nov 04 01:59:56.977626 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:56.977606    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="929d4558-6de3-4bf9-894a-169498b5ed7b" containerName="revoke-keys"
Nov 04 01:59:56.977721 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:56.977664    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="929d4558-6de3-4bf9-894a-169498b5ed7b" containerName="revoke-keys"
Nov 04 01:59:56.977721 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:56.977677    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1501413-7507-4f1d-ae16-de82aed103ea" containerName="main"
Nov 04 01:59:56.980887 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:56.980867    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.028552 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.028515    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"]
Nov 04 01:59:57.032715 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.032691    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"]
Nov 04 01:59:57.032848 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.032821    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.036190 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.036170    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.043045 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.043020    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/437e07b5-3594-4bd4-b035-49047e446a61-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.043191 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.043173    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/437e07b5-3594-4bd4-b035-49047e446a61-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.043248 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.043205    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfbbc\" (UniqueName: \"kubernetes.io/projected/437e07b5-3594-4bd4-b035-49047e446a61-kube-api-access-rfbbc\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.043318 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.043298    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/437e07b5-3594-4bd4-b035-49047e446a61-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.043389 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.043346    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/437e07b5-3594-4bd4-b035-49047e446a61-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.043486 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.043421    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/437e07b5-3594-4bd4-b035-49047e446a61-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.043551 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.043481    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/437e07b5-3594-4bd4-b035-49047e446a61-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.043551 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.043518    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/437e07b5-3594-4bd4-b035-49047e446a61-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.043643 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.043603    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/437e07b5-3594-4bd4-b035-49047e446a61-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.047665 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.047630    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"]
Nov 04 01:59:57.052094 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.052074    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.074239 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.074216    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-plld6\""
Nov 04 01:59:57.075317 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.075292    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt" podUID="f051c9f2-e56c-4ced-b88d-b85747f3254f" containerName="payload-pre-processing" containerID="cri-o://d044c4c1dfc6471107413392659fa3c409bdf6f836e56b5c1c26568b7f0f6f35" gracePeriod=25
Nov 04 01:59:57.084702 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.084673    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"]
Nov 04 01:59:57.138714 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.138680    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"]
Nov 04 01:59:57.144497 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.144440    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/437e07b5-3594-4bd4-b035-49047e446a61-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.144737 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.144707    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/437e07b5-3594-4bd4-b035-49047e446a61-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.144746    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/437e07b5-3594-4bd4-b035-49047e446a61-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.144779    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2376946a-54d9-4819-b016-d91a718df43a-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.144855    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/437e07b5-3594-4bd4-b035-49047e446a61-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.145430    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/437e07b5-3594-4bd4-b035-49047e446a61-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.145515    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2376946a-54d9-4819-b016-d91a718df43a-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.145554    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41006a52-5c19-43e2-ba93-a2b1ca062162-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.145597    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2376946a-54d9-4819-b016-d91a718df43a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.145621    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97871e88-cdc1-43ca-aca2-e3bbd128b086-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.145650    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/437e07b5-3594-4bd4-b035-49047e446a61-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.145675    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2376946a-54d9-4819-b016-d91a718df43a-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.145704    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2376946a-54d9-4819-b016-d91a718df43a-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.145735    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x2q7\" (UniqueName: \"kubernetes.io/projected/2376946a-54d9-4819-b016-d91a718df43a-kube-api-access-4x2q7\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.145763    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41006a52-5c19-43e2-ba93-a2b1ca062162-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.145789    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgppv\" (UniqueName: \"kubernetes.io/projected/41006a52-5c19-43e2-ba93-a2b1ca062162-kube-api-access-pgppv\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.145812    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2376946a-54d9-4819-b016-d91a718df43a-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.145839    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97871e88-cdc1-43ca-aca2-e3bbd128b086-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.145872    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbkjx\" (UniqueName: \"kubernetes.io/projected/97871e88-cdc1-43ca-aca2-e3bbd128b086-kube-api-access-pbkjx\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.145925    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41006a52-5c19-43e2-ba93-a2b1ca062162-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.145948    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97871e88-cdc1-43ca-aca2-e3bbd128b086-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.145982    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41006a52-5c19-43e2-ba93-a2b1ca062162-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.146007    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"]
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.146012    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41006a52-5c19-43e2-ba93-a2b1ca062162-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.146134    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97871e88-cdc1-43ca-aca2-e3bbd128b086-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.146161    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2376946a-54d9-4819-b016-d91a718df43a-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.146195    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/437e07b5-3594-4bd4-b035-49047e446a61-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.146221    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfbbc\" (UniqueName: \"kubernetes.io/projected/437e07b5-3594-4bd4-b035-49047e446a61-kube-api-access-rfbbc\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.146256    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41006a52-5c19-43e2-ba93-a2b1ca062162-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.146291    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41006a52-5c19-43e2-ba93-a2b1ca062162-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.146594 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.146296    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/437e07b5-3594-4bd4-b035-49047e446a61-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.147584 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.146884    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/437e07b5-3594-4bd4-b035-49047e446a61-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.147584 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.146951    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/437e07b5-3594-4bd4-b035-49047e446a61-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.147584 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.147048    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2376946a-54d9-4819-b016-d91a718df43a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.147584 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.147120    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/437e07b5-3594-4bd4-b035-49047e446a61-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.147584 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.147163    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97871e88-cdc1-43ca-aca2-e3bbd128b086-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.147584 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.147288    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/437e07b5-3594-4bd4-b035-49047e446a61-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.147584 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.147345    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/437e07b5-3594-4bd4-b035-49047e446a61-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.147584 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.147537    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97871e88-cdc1-43ca-aca2-e3bbd128b086-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.147943 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.147624    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41006a52-5c19-43e2-ba93-a2b1ca062162-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.147943 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.147711    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97871e88-cdc1-43ca-aca2-e3bbd128b086-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.147943 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.147842    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97871e88-cdc1-43ca-aca2-e3bbd128b086-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.150159 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.150128    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/437e07b5-3594-4bd4-b035-49047e446a61-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.150266 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.150216    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/437e07b5-3594-4bd4-b035-49047e446a61-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.213350 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.213318    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/437e07b5-3594-4bd4-b035-49047e446a61-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.213556 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.213522    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfbbc\" (UniqueName: \"kubernetes.io/projected/437e07b5-3594-4bd4-b035-49047e446a61-kube-api-access-rfbbc\") pod \"maas-default-gateway-openshift-default-778b7b5c47-s8k5g\" (UID: \"437e07b5-3594-4bd4-b035-49047e446a61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.235358 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.235190    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"]
Nov 04 01:59:57.249018 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.248984    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2376946a-54d9-4819-b016-d91a718df43a-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.249191 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.249041    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41006a52-5c19-43e2-ba93-a2b1ca062162-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.249191 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.249076    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2376946a-54d9-4819-b016-d91a718df43a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.249191 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.249100    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97871e88-cdc1-43ca-aca2-e3bbd128b086-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.249191 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.249120    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2376946a-54d9-4819-b016-d91a718df43a-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.249191 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.249144    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2376946a-54d9-4819-b016-d91a718df43a-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.249191 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.249171    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x2q7\" (UniqueName: \"kubernetes.io/projected/2376946a-54d9-4819-b016-d91a718df43a-kube-api-access-4x2q7\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.249530 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.249434    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2376946a-54d9-4819-b016-d91a718df43a-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.249784 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.249762    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41006a52-5c19-43e2-ba93-a2b1ca062162-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.249998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.249978    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2376946a-54d9-4819-b016-d91a718df43a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.249998 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.249991    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2376946a-54d9-4819-b016-d91a718df43a-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.250189 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.249204    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41006a52-5c19-43e2-ba93-a2b1ca062162-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.250240 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250203    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgppv\" (UniqueName: \"kubernetes.io/projected/41006a52-5c19-43e2-ba93-a2b1ca062162-kube-api-access-pgppv\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.250428 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250409    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2376946a-54d9-4819-b016-d91a718df43a-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.250517 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250472    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97871e88-cdc1-43ca-aca2-e3bbd128b086-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.250517 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250506    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbkjx\" (UniqueName: \"kubernetes.io/projected/97871e88-cdc1-43ca-aca2-e3bbd128b086-kube-api-access-pbkjx\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.250625 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250553    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41006a52-5c19-43e2-ba93-a2b1ca062162-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.250625 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250577    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97871e88-cdc1-43ca-aca2-e3bbd128b086-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.250625 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250611    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41006a52-5c19-43e2-ba93-a2b1ca062162-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.250777 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250644    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41006a52-5c19-43e2-ba93-a2b1ca062162-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.250777 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250699    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97871e88-cdc1-43ca-aca2-e3bbd128b086-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.250777 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250724    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2376946a-54d9-4819-b016-d91a718df43a-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.250777 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250762    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41006a52-5c19-43e2-ba93-a2b1ca062162-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.250972 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250795    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41006a52-5c19-43e2-ba93-a2b1ca062162-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.250972 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250804    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41006a52-5c19-43e2-ba93-a2b1ca062162-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.250972 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250825    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2376946a-54d9-4819-b016-d91a718df43a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.250972 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250853    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97871e88-cdc1-43ca-aca2-e3bbd128b086-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.250972 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250892    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97871e88-cdc1-43ca-aca2-e3bbd128b086-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.250972 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250923    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41006a52-5c19-43e2-ba93-a2b1ca062162-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.250972 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250950    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97871e88-cdc1-43ca-aca2-e3bbd128b086-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.251296 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.250987    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97871e88-cdc1-43ca-aca2-e3bbd128b086-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.251296 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.251039    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2376946a-54d9-4819-b016-d91a718df43a-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.253283 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.251798    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97871e88-cdc1-43ca-aca2-e3bbd128b086-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.253283 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.251903    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2376946a-54d9-4819-b016-d91a718df43a-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.253283 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.252145    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2376946a-54d9-4819-b016-d91a718df43a-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.253283 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.252352    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97871e88-cdc1-43ca-aca2-e3bbd128b086-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.253283 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.252539    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41006a52-5c19-43e2-ba93-a2b1ca062162-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.253283 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.252745    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41006a52-5c19-43e2-ba93-a2b1ca062162-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.253878 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.253846    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97871e88-cdc1-43ca-aca2-e3bbd128b086-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.254309 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.254289    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97871e88-cdc1-43ca-aca2-e3bbd128b086-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.254535 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.254517    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97871e88-cdc1-43ca-aca2-e3bbd128b086-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.255111 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.255071    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41006a52-5c19-43e2-ba93-a2b1ca062162-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.255191 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.255110    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41006a52-5c19-43e2-ba93-a2b1ca062162-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.257379 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.257355    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2376946a-54d9-4819-b016-d91a718df43a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.262894 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.262868    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97871e88-cdc1-43ca-aca2-e3bbd128b086-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.264036 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.263948    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2376946a-54d9-4819-b016-d91a718df43a-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.264036 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.264004    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41006a52-5c19-43e2-ba93-a2b1ca062162-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.264149 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.264028    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97871e88-cdc1-43ca-aca2-e3bbd128b086-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.293619 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.293591    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:57.313114 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.313079    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x2q7\" (UniqueName: \"kubernetes.io/projected/2376946a-54d9-4819-b016-d91a718df43a-kube-api-access-4x2q7\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.314436 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.314412    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2376946a-54d9-4819-b016-d91a718df43a-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-m59rh\" (UID: \"2376946a-54d9-4819-b016-d91a718df43a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.314563 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.314504    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgppv\" (UniqueName: \"kubernetes.io/projected/41006a52-5c19-43e2-ba93-a2b1ca062162-kube-api-access-pgppv\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.314871 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.314849    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbkjx\" (UniqueName: \"kubernetes.io/projected/97871e88-cdc1-43ca-aca2-e3bbd128b086-kube-api-access-pbkjx\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.316730 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.316709    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41006a52-5c19-43e2-ba93-a2b1ca062162-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-qnxqv\" (UID: \"41006a52-5c19-43e2-ba93-a2b1ca062162\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.318665 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.318646    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97871e88-cdc1-43ca-aca2-e3bbd128b086-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-5x8z8\" (UID: \"97871e88-cdc1-43ca-aca2-e3bbd128b086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.348003 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.347563    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:57.359431 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.354203    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:57.363509 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.363488    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:57.674799 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.674773    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"]
Nov 04 01:59:57.675698 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:59:57.675668    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod437e07b5_3594_4bd4_b035_49047e446a61.slice/crio-44323ca2dcaee1e721033146a7f61da7e6109c1e4d2f23fef88f1b4481842430 WatchSource:0}: Error finding container 44323ca2dcaee1e721033146a7f61da7e6109c1e4d2f23fef88f1b4481842430: Status 404 returned error can't find the container with id 44323ca2dcaee1e721033146a7f61da7e6109c1e4d2f23fef88f1b4481842430
Nov 04 01:59:57.677757 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.677722    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:59:57.677835 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.677790    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:59:57.677868 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.677840    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:59:57.779258 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.779232    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"]
Nov 04 01:59:57.780247 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:59:57.780202    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41006a52_5c19_43e2_ba93_a2b1ca062162.slice/crio-68a0f7add70473bf8959cd5f036a64b66e33be991b8134799ee39aa9e591e3a0 WatchSource:0}: Error finding container 68a0f7add70473bf8959cd5f036a64b66e33be991b8134799ee39aa9e591e3a0: Status 404 returned error can't find the container with id 68a0f7add70473bf8959cd5f036a64b66e33be991b8134799ee39aa9e591e3a0
Nov 04 01:59:57.782611 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.782571    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:59:57.782708 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.782692    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:59:57.782748 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.782737    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:59:57.827663 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.827637    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"]
Nov 04 01:59:57.828542 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:59:57.828506    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97871e88_cdc1_43ca_aca2_e3bbd128b086.slice/crio-5e9abd4452c35f6effc4dedb3b931bcade15adf9af95d5501546dd8c677b8ba2 WatchSource:0}: Error finding container 5e9abd4452c35f6effc4dedb3b931bcade15adf9af95d5501546dd8c677b8ba2: Status 404 returned error can't find the container with id 5e9abd4452c35f6effc4dedb3b931bcade15adf9af95d5501546dd8c677b8ba2
Nov 04 01:59:57.830880 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.830861    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"]
Nov 04 01:59:57.831107 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.831076    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:59:57.831182 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.831151    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:59:57.831249 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.831194    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:59:57.836128 ip-10-0-129-138 kubenswrapper[6116]: W1104 01:59:57.835345    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2376946a_54d9_4819_b016_d91a718df43a.slice/crio-072b60c5be6f170693b2d4e09202f4e21c95f3b1aa1ae68b585449ce216c0365 WatchSource:0}: Error finding container 072b60c5be6f170693b2d4e09202f4e21c95f3b1aa1ae68b585449ce216c0365: Status 404 returned error can't find the container with id 072b60c5be6f170693b2d4e09202f4e21c95f3b1aa1ae68b585449ce216c0365
Nov 04 01:59:57.841246 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.841208    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:59:57.841336 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.841297    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:59:57.841375 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:57.841345    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 01:59:58.025051 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:58.024969    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh" podUID="aaa5bb34-fdba-497a-b58b-9ca549fa3d18" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:59:58.025051 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:58.025035    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt" podUID="f051c9f2-e56c-4ced-b88d-b85747f3254f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:59:58.070828 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:58.070790    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 01:59:58.402799 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:58.402762    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8" event={"ID":"97871e88-cdc1-43ca-aca2-e3bbd128b086","Type":"ContainerStarted","Data":"df10ff6fa7af9c0d3812dc1314402ebe48b7cdb904ccc466782637d0c2faead7"}
Nov 04 01:59:58.403006 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:58.402835    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8" event={"ID":"97871e88-cdc1-43ca-aca2-e3bbd128b086","Type":"ContainerStarted","Data":"5e9abd4452c35f6effc4dedb3b931bcade15adf9af95d5501546dd8c677b8ba2"}
Nov 04 01:59:58.407001 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:58.406236    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv" event={"ID":"41006a52-5c19-43e2-ba93-a2b1ca062162","Type":"ContainerStarted","Data":"8361d0fbfe0270724ce2b4a476fbc4c4f135b9151f38b81d37396d14ae382652"}
Nov 04 01:59:58.407001 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:58.406302    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv" event={"ID":"41006a52-5c19-43e2-ba93-a2b1ca062162","Type":"ContainerStarted","Data":"68a0f7add70473bf8959cd5f036a64b66e33be991b8134799ee39aa9e591e3a0"}
Nov 04 01:59:58.409473 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:58.409401    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh" event={"ID":"2376946a-54d9-4819-b016-d91a718df43a","Type":"ContainerStarted","Data":"87b51675d737cf6da296cb9b8ecaddaf69b6b99bfeb0c655971bb290da7211af"}
Nov 04 01:59:58.409784 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:58.409686    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh" event={"ID":"2376946a-54d9-4819-b016-d91a718df43a","Type":"ContainerStarted","Data":"072b60c5be6f170693b2d4e09202f4e21c95f3b1aa1ae68b585449ce216c0365"}
Nov 04 01:59:58.412665 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:58.412644    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g" event={"ID":"437e07b5-3594-4bd4-b035-49047e446a61","Type":"ContainerStarted","Data":"68260e6e31409f7dedb98cc7b9dc3d1bb06042c9de3a42a24b97c9800b8a61ce"}
Nov 04 01:59:58.412851 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:58.412829    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g" event={"ID":"437e07b5-3594-4bd4-b035-49047e446a61","Type":"ContainerStarted","Data":"44323ca2dcaee1e721033146a7f61da7e6109c1e4d2f23fef88f1b4481842430"}
Nov 04 01:59:58.540312 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:58.538805    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8" podStartSLOduration=2.538784187 podStartE2EDuration="2.538784187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:59:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:59:58.534610341 +0000 UTC m=+1483.655887868" watchObservedRunningTime="2026-11-04 01:59:58.538784187 +0000 UTC m=+1483.660061714"
Nov 04 01:59:58.686544 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:58.678226    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g" podStartSLOduration=2.678187814 podStartE2EDuration="2.678187814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:59:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:59:58.655422356 +0000 UTC m=+1483.776699884" watchObservedRunningTime="2026-11-04 01:59:58.678187814 +0000 UTC m=+1483.799465344"
Nov 04 01:59:58.959869 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:58.959747    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh" podStartSLOduration=2.959728353 podStartE2EDuration="2.959728353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:59:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:59:58.958019777 +0000 UTC m=+1484.079297304" watchObservedRunningTime="2026-11-04 01:59:58.959728353 +0000 UTC m=+1484.081005880"
Nov 04 01:59:58.962110 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:58.962054    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv" podStartSLOduration=2.9620415060000003 podStartE2EDuration="2.962041506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:59:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:59:58.818371484 +0000 UTC m=+1483.939649010" watchObservedRunningTime="2026-11-04 01:59:58.962041506 +0000 UTC m=+1484.083319035"
Nov 04 01:59:59.302098 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:59.301553    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 01:59:59.302098 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:59.301719    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-s8k5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Nov 04 01:59:59.302098 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:59.301767    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g" podUID="437e07b5-3594-4bd4-b035-49047e446a61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Nov 04 01:59:59.352549 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:59.352513    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 01:59:59.353933 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:59.353898    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-qnxqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Nov 04 01:59:59.354211 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:59.354171    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv" podUID="41006a52-5c19-43e2-ba93-a2b1ca062162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Nov 04 01:59:59.355568 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:59.355502    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 01:59:59.359956 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:59.359542    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5x8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Nov 04 01:59:59.359956 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:59.359590    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8" podUID="97871e88-cdc1-43ca-aca2-e3bbd128b086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Nov 04 01:59:59.365101 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:59.364905    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 01:59:59.365101 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:59.365008    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-m59rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Nov 04 01:59:59.365101 ip-10-0-129-138 kubenswrapper[6116]: I1104 01:59:59.365045    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh" podUID="2376946a-54d9-4819-b016-d91a718df43a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Nov 04 02:00:00.298481 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:00.294835    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-s8k5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Nov 04 02:00:00.307484 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:00.300577    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g" podUID="437e07b5-3594-4bd4-b035-49047e446a61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Nov 04 02:00:00.382491 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:00.381618    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5x8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Nov 04 02:00:00.382491 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:00.381669    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8" podUID="97871e88-cdc1-43ca-aca2-e3bbd128b086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Nov 04 02:00:00.382491 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:00.381863    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-qnxqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Nov 04 02:00:00.382491 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:00.381886    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv" podUID="41006a52-5c19-43e2-ba93-a2b1ca062162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Nov 04 02:00:00.382491 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:00.381937    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-m59rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Nov 04 02:00:00.382491 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:00.381957    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh" podUID="2376946a-54d9-4819-b016-d91a718df43a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Nov 04 02:00:01.319473 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:01.298916    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-s8k5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Nov 04 02:00:01.319473 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:01.299000    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g" podUID="437e07b5-3594-4bd4-b035-49047e446a61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Nov 04 02:00:01.365480 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:01.364988    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-m59rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Nov 04 02:00:01.365480 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:01.365041    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh" podUID="2376946a-54d9-4819-b016-d91a718df43a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Nov 04 02:00:01.365480 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:01.365112    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-qnxqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Nov 04 02:00:01.365480 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:01.365134    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv" podUID="41006a52-5c19-43e2-ba93-a2b1ca062162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Nov 04 02:00:01.365480 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:01.365184    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5x8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Nov 04 02:00:01.365480 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:01.365203    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8" podUID="97871e88-cdc1-43ca-aca2-e3bbd128b086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Nov 04 02:00:02.085474 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.082709    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr"]
Nov 04 02:00:02.091691 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.091653    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d68d3e6-b7sjr"]
Nov 04 02:00:02.130902 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.130869    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"]
Nov 04 02:00:02.149836 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.149805    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.205540 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.205489    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-7af38e-openshift-default-dockercfg-pbl8c\""
Nov 04 02:00:02.208383 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.208347    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"]
Nov 04 02:00:02.329578 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.329523    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-s8k5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Nov 04 02:00:02.350560 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.342616    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g" podUID="437e07b5-3594-4bd4-b035-49047e446a61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Nov 04 02:00:02.366499 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.353567    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-qnxqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Nov 04 02:00:02.366499 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.353644    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv" podUID="41006a52-5c19-43e2-ba93-a2b1ca062162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Nov 04 02:00:02.392150 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.391625    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5x8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Nov 04 02:00:02.392150 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.391705    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8" podUID="97871e88-cdc1-43ca-aca2-e3bbd128b086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Nov 04 02:00:02.392150 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.392057    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-m59rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Nov 04 02:00:02.392150 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.392089    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh" podUID="2376946a-54d9-4819-b016-d91a718df43a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Nov 04 02:00:02.415580 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.408591    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-workload-certs\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.419599 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.419567    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-podinfo\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.486170 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.486077    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-envoy\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.486520 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.486496    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-credential-socket\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.486761 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.486743    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bee69a37-fadb-462f-9514-e8cb641fe4b1-istiod-ca-cert\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.486973 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.486936    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txcmd\" (UniqueName: \"kubernetes.io/projected/bee69a37-fadb-462f-9514-e8cb641fe4b1-kube-api-access-txcmd\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.487184 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.487165    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-data\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.487505 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.487488    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-token\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.513358 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.508562    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-workload-socket\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.611116 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.611006    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-data\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.611472 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.611425    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-token\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.611629 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.611612    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-workload-socket\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.611764 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.611749    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-workload-certs\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.611870 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.611856    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-podinfo\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.611998 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.611985    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-envoy\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.612143 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.612105    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-credential-socket\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.612260 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.612246    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bee69a37-fadb-462f-9514-e8cb641fe4b1-istiod-ca-cert\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.612360 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.612347    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txcmd\" (UniqueName: \"kubernetes.io/projected/bee69a37-fadb-462f-9514-e8cb641fe4b1-kube-api-access-txcmd\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.624475 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.613193    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-data\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.624709 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.622175    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-workload-socket\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.624802 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.622402    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-workload-certs\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.625260 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.625237    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-credential-socket\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.626368 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.626342    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bee69a37-fadb-462f-9514-e8cb641fe4b1-istiod-ca-cert\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.644480 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.644115    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-podinfo\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.656824 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.656784    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-envoy\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.684202 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.684159    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txcmd\" (UniqueName: \"kubernetes.io/projected/bee69a37-fadb-462f-9514-e8cb641fe4b1-kube-api-access-txcmd\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.700374 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.700339    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-token\") pod \"e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") " pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:02.781836 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:02.781797    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:03.058055 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:03.057953    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf7d56e8-d90e-4557-b72d-a3ac56015867" path="/var/lib/kubelet/pods/cf7d56e8-d90e-4557-b72d-a3ac56015867/volumes"
Nov 04 02:00:03.305216 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:03.305176    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-s8k5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Nov 04 02:00:03.305378 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:03.305256    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g" podUID="437e07b5-3594-4bd4-b035-49047e446a61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Nov 04 02:00:03.420514 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:03.373797    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5x8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Nov 04 02:00:03.420514 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:03.373855    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8" podUID="97871e88-cdc1-43ca-aca2-e3bbd128b086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Nov 04 02:00:03.420514 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:03.373929    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-qnxqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Nov 04 02:00:03.420514 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:03.373952    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv" podUID="41006a52-5c19-43e2-ba93-a2b1ca062162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Nov 04 02:00:03.420514 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:03.385663    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-m59rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Nov 04 02:00:03.420514 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:03.385730    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh" podUID="2376946a-54d9-4819-b016-d91a718df43a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Nov 04 02:00:03.867282 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:03.867175    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"]
Nov 04 02:00:03.960785 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:03.959984    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 02:00:03.960785 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:03.960098    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 02:00:03.960785 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:03.960142    6116 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Nov 04 02:00:04.059596 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:04.059506    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5"]
Nov 04 02:00:04.061172 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:04.061133    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-da8cb04f-nzhc5"]
Nov 04 02:00:04.303028 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:04.302994    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-s8k5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Nov 04 02:00:04.303285 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:04.303260    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g" podUID="437e07b5-3594-4bd4-b035-49047e446a61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Nov 04 02:00:04.351586 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:04.351431    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-qnxqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Nov 04 02:00:04.351586 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:04.351521    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv" podUID="41006a52-5c19-43e2-ba93-a2b1ca062162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Nov 04 02:00:04.357584 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:04.357439    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5x8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Nov 04 02:00:04.357584 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:04.357527    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8" podUID="97871e88-cdc1-43ca-aca2-e3bbd128b086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Nov 04 02:00:04.365692 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:04.365581    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-m59rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Nov 04 02:00:04.365692 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:04.365636    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh" podUID="2376946a-54d9-4819-b016-d91a718df43a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Nov 04 02:00:04.746488 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:04.745679    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w" event={"ID":"bee69a37-fadb-462f-9514-e8cb641fe4b1","Type":"ContainerStarted","Data":"0f4885944b2769a4c5dccbbe90fc16993e57fff996b7934e4c8d5e87cb8bf6d3"}
Nov 04 02:00:04.746488 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:04.745750    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w" event={"ID":"bee69a37-fadb-462f-9514-e8cb641fe4b1","Type":"ContainerStarted","Data":"67f8027182f6607c79f52c3fa385d3bc4345b65617b0f2e2329c99dd8f756127"}
Nov 04 02:00:04.784666 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:04.784571    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:04.881149 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:04.881080    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w" podStartSLOduration=2.881060102 podStartE2EDuration="2.881060102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:00:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:00:04.875441237 +0000 UTC m=+1489.996718764" watchObservedRunningTime="2026-11-04 02:00:04.881060102 +0000 UTC m=+1490.002337628"
Nov 04 02:00:05.057534 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:05.047213    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="843e5f1b-1c67-4b12-b617-1b4f293baa0c" path="/var/lib/kubelet/pods/843e5f1b-1c67-4b12-b617-1b4f293baa0c/volumes"
Nov 04 02:00:05.299068 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:05.299010    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-s8k5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Nov 04 02:00:05.299367 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:05.299315    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g" podUID="437e07b5-3594-4bd4-b035-49047e446a61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Nov 04 02:00:05.356081 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:05.355988    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-qnxqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Nov 04 02:00:05.356377 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:05.356350    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv" podUID="41006a52-5c19-43e2-ba93-a2b1ca062162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Nov 04 02:00:05.356530 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:05.356292    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5x8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Nov 04 02:00:05.356676 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:05.356658    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8" podUID="97871e88-cdc1-43ca-aca2-e3bbd128b086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Nov 04 02:00:05.366259 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:05.366229    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-m59rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Nov 04 02:00:05.366534 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:05.366509    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh" podUID="2376946a-54d9-4819-b016-d91a718df43a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Nov 04 02:00:05.786479 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:05.786147    6116 patch_prober.go:29] interesting pod/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 04 02:00:05.786479 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:05.786220    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w" podUID="bee69a37-fadb-462f-9514-e8cb641fe4b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 04 02:00:06.295518 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:06.295475    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-s8k5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Nov 04 02:00:06.295777 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:06.295749    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g" podUID="437e07b5-3594-4bd4-b035-49047e446a61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Nov 04 02:00:06.350478 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:06.349840    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-qnxqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Nov 04 02:00:06.350478 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:06.349904    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv" podUID="41006a52-5c19-43e2-ba93-a2b1ca062162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Nov 04 02:00:06.365475 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:06.358779    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5x8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Nov 04 02:00:06.365475 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:06.358840    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8" podUID="97871e88-cdc1-43ca-aca2-e3bbd128b086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Nov 04 02:00:06.365475 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:06.365168    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-m59rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Nov 04 02:00:06.365475 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:06.365217    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh" podUID="2376946a-54d9-4819-b016-d91a718df43a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Nov 04 02:00:06.788474 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:06.787715    6116 patch_prober.go:29] interesting pod/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 04 02:00:06.788474 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:06.787778    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w" podUID="bee69a37-fadb-462f-9514-e8cb641fe4b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 04 02:00:07.784474 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:07.783487    6116 patch_prober.go:29] interesting pod/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 04 02:00:07.784474 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:07.783547    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w" podUID="bee69a37-fadb-462f-9514-e8cb641fe4b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 04 02:00:08.133542 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.132420    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"]
Nov 04 02:00:08.133542 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.132662    6116 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w" podUID="bee69a37-fadb-462f-9514-e8cb641fe4b1" containerName="istio-proxy" containerID="cri-o://0f4885944b2769a4c5dccbbe90fc16993e57fff996b7934e4c8d5e87cb8bf6d3" gracePeriod=30
Nov 04 02:00:08.169507 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.169394    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt" podUID="f051c9f2-e56c-4ced-b88d-b85747f3254f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:00:08.208489 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.191585    6116 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh" podUID="aaa5bb34-fdba-497a-b58b-9ca549fa3d18" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:00:08.301579 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.301522    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-s8k5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:00:08.301881 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.301828    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g" podUID="437e07b5-3594-4bd4-b035-49047e446a61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:00:08.391531 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.391224    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-qnxqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:00:08.391531 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.391351    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv" podUID="41006a52-5c19-43e2-ba93-a2b1ca062162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:00:08.415768 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.414070    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 02:00:08.415768 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.415348    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 02:00:08.418594 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.418558    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-m59rh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:00:08.418822 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.418795    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh" podUID="2376946a-54d9-4819-b016-d91a718df43a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:00:08.419278 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.419255    6116 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-5x8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:00:08.419430 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.419409    6116 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8" podUID="97871e88-cdc1-43ca-aca2-e3bbd128b086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:00:08.431517 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.427685    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g"
Nov 04 02:00:08.677472 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.676883    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 02:00:08.711512 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.691802    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 02:00:08.712105 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.712083    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh"
Nov 04 02:00:08.987279 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.986865    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 02:00:08.994500 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.988180    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 02:00:08.994500 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:08.993690    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8"
Nov 04 02:00:09.037513 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:09.037483    6116 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 02:00:09.038518 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:09.038499    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 02:00:09.039890 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:09.039871    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv"
Nov 04 02:00:11.319437 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:11.319405    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-mzk5s_66494911-a785-4301-9401-c5ba28818edb/limitador/0.log"
Nov 04 02:00:11.621702 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:11.621603    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-jt25s_451d239a-b06e-4ca3-af9b-7affd25cb5ff/manager/0.log"
Nov 04 02:00:11.991331 ip-10-0-129-138 kubenswrapper[6116]: E1104 02:00:11.991287    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 02:00:12.913049 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:12.913002    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6d4cf6d5f-9wgr6_e62dea92-10fa-4569-9477-2010e4897b58/manager/0.log"
Nov 04 02:00:13.047282 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:13.047246    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd"]
Nov 04 02:00:13.096765 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:13.096732    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-166efb82-hrvgd"]
Nov 04 02:00:15.027641 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:15.027606    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33bf4adf-572f-4369-9f99-b685fb65f9e0" path="/var/lib/kubelet/pods/33bf4adf-572f-4369-9f99-b685fb65f9e0/volumes"
Nov 04 02:00:15.278224 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:15.278148    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/prometheus/0.log"
Nov 04 02:00:15.280568 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:15.280548    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/prometheus/0.log"
Nov 04 02:00:15.291142 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:15.291119    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6lfbv_192e2fec-fefb-4bc5-9516-ed7464963a10/ovn-acl-logging/0.log"
Nov 04 02:00:15.292872 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:15.292856    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6lfbv_192e2fec-fefb-4bc5-9516-ed7464963a10/ovn-acl-logging/0.log"
Nov 04 02:00:15.444685 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:15.444658    6116 scope.go:122] "RemoveContainer" containerID="62205101a5b80c7bc68a323c57017a6dadf6cb8e8e734567ccf94d44666c9fc6"
Nov 04 02:00:15.748070 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:15.748045    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-lgdkj_74baa758-c88e-4814-926d-a59456774d5f/server/0.log"
Nov 04 02:00:16.177920 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:16.177894    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-l5vpc_61e56398-a014-42ad-89e9-dbd7fa34ec59/manager/0.log"
Nov 04 02:00:16.776385 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:16.776357    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-4qznm_d6ffa44d-084d-4df2-962c-d1f8cc30507e/proxy/0.log"
Nov 04 02:00:17.214149 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.214121    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh"
Nov 04 02:00:17.266525 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.265957    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aaa5bb34-fdba-497a-b58b-9ca549fa3d18-kube-api-access-6d8t2\" (UniqueName: \"kubernetes.io/projected/aaa5bb34-fdba-497a-b58b-9ca549fa3d18-kube-api-access-6d8t2\") pod \"aaa5bb34-fdba-497a-b58b-9ca549fa3d18\" (UID: \"aaa5bb34-fdba-497a-b58b-9ca549fa3d18\") "
Nov 04 02:00:17.266525 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.266045    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aaa5bb34-fdba-497a-b58b-9ca549fa3d18-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aaa5bb34-fdba-497a-b58b-9ca549fa3d18-plugins-config-volume\") pod \"aaa5bb34-fdba-497a-b58b-9ca549fa3d18\" (UID: \"aaa5bb34-fdba-497a-b58b-9ca549fa3d18\") "
Nov 04 02:00:17.267007 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.266958    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaa5bb34-fdba-497a-b58b-9ca549fa3d18-plugins-config-volume" pod "aaa5bb34-fdba-497a-b58b-9ca549fa3d18" (UID: "aaa5bb34-fdba-497a-b58b-9ca549fa3d18"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:00:17.269681 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.269640    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaa5bb34-fdba-497a-b58b-9ca549fa3d18-kube-api-access-6d8t2" pod "aaa5bb34-fdba-497a-b58b-9ca549fa3d18" (UID: "aaa5bb34-fdba-497a-b58b-9ca549fa3d18"). InnerVolumeSpecName "kube-api-access-6d8t2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:00:17.314659 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.314632    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:17.353363 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.353332    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt"
Nov 04 02:00:17.367492 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.367468    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-data\") pod \"bee69a37-fadb-462f-9514-e8cb641fe4b1\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") "
Nov 04 02:00:17.367603 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.367532    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-token\" (UniqueName: \"kubernetes.io/projected/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-token\") pod \"bee69a37-fadb-462f-9514-e8cb641fe4b1\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") "
Nov 04 02:00:17.367603 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.367592    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-envoy\") pod \"bee69a37-fadb-462f-9514-e8cb641fe4b1\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") "
Nov 04 02:00:17.367684 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.367666    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-workload-socket\") pod \"bee69a37-fadb-462f-9514-e8cb641fe4b1\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") "
Nov 04 02:00:17.367727 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.367696    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bee69a37-fadb-462f-9514-e8cb641fe4b1-kube-api-access-txcmd\" (UniqueName: \"kubernetes.io/projected/bee69a37-fadb-462f-9514-e8cb641fe4b1-kube-api-access-txcmd\") pod \"bee69a37-fadb-462f-9514-e8cb641fe4b1\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") "
Nov 04 02:00:17.367727 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.367705    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-data" pod "bee69a37-fadb-462f-9514-e8cb641fe4b1" (UID: "bee69a37-fadb-462f-9514-e8cb641fe4b1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:17.367818 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.367730    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bee69a37-fadb-462f-9514-e8cb641fe4b1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bee69a37-fadb-462f-9514-e8cb641fe4b1-istiod-ca-cert\") pod \"bee69a37-fadb-462f-9514-e8cb641fe4b1\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") "
Nov 04 02:00:17.367818 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.367771    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-workload-certs\") pod \"bee69a37-fadb-462f-9514-e8cb641fe4b1\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") "
Nov 04 02:00:17.367919 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.367822    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-podinfo\") pod \"bee69a37-fadb-462f-9514-e8cb641fe4b1\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") "
Nov 04 02:00:17.367919 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.367854    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-credential-socket\") pod \"bee69a37-fadb-462f-9514-e8cb641fe4b1\" (UID: \"bee69a37-fadb-462f-9514-e8cb641fe4b1\") "
Nov 04 02:00:17.368018 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.367953    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-workload-socket" pod "bee69a37-fadb-462f-9514-e8cb641fe4b1" (UID: "bee69a37-fadb-462f-9514-e8cb641fe4b1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:17.368146 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.368119    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-workload-certs" pod "bee69a37-fadb-462f-9514-e8cb641fe4b1" (UID: "bee69a37-fadb-462f-9514-e8cb641fe4b1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:17.368273 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.368258    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6d8t2\" (UniqueName: \"kubernetes.io/projected/aaa5bb34-fdba-497a-b58b-9ca549fa3d18-kube-api-access-6d8t2\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:17.368515 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.368276    6116 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-workload-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:17.368515 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.368317    6116 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-workload-certs\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:17.368515 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.368333    6116 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aaa5bb34-fdba-497a-b58b-9ca549fa3d18-plugins-config-volume\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:17.368515 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.368345    6116 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-data\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:17.368722 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.368550    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-credential-socket" pod "bee69a37-fadb-462f-9514-e8cb641fe4b1" (UID: "bee69a37-fadb-462f-9514-e8cb641fe4b1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:17.368780 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.368712    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bee69a37-fadb-462f-9514-e8cb641fe4b1-istiod-ca-cert" pod "bee69a37-fadb-462f-9514-e8cb641fe4b1" (UID: "bee69a37-fadb-462f-9514-e8cb641fe4b1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:00:17.370017 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.369993    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bee69a37-fadb-462f-9514-e8cb641fe4b1-kube-api-access-txcmd" pod "bee69a37-fadb-462f-9514-e8cb641fe4b1" (UID: "bee69a37-fadb-462f-9514-e8cb641fe4b1"). InnerVolumeSpecName "kube-api-access-txcmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:00:17.370017 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.370006    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-token" pod "bee69a37-fadb-462f-9514-e8cb641fe4b1" (UID: "bee69a37-fadb-462f-9514-e8cb641fe4b1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:00:17.370186 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.370172    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-envoy" pod "bee69a37-fadb-462f-9514-e8cb641fe4b1" (UID: "bee69a37-fadb-462f-9514-e8cb641fe4b1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:00:17.370371 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.370347    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-podinfo" pod "bee69a37-fadb-462f-9514-e8cb641fe4b1" (UID: "bee69a37-fadb-462f-9514-e8cb641fe4b1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:00:17.469153 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.469124    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f051c9f2-e56c-4ced-b88d-b85747f3254f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f051c9f2-e56c-4ced-b88d-b85747f3254f-plugins-config-volume\") pod \"f051c9f2-e56c-4ced-b88d-b85747f3254f\" (UID: \"f051c9f2-e56c-4ced-b88d-b85747f3254f\") "
Nov 04 02:00:17.469335 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.469246    6116 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f051c9f2-e56c-4ced-b88d-b85747f3254f-kube-api-access-9qdwj\" (UniqueName: \"kubernetes.io/projected/f051c9f2-e56c-4ced-b88d-b85747f3254f-kube-api-access-9qdwj\") pod \"f051c9f2-e56c-4ced-b88d-b85747f3254f\" (UID: \"f051c9f2-e56c-4ced-b88d-b85747f3254f\") "
Nov 04 02:00:17.469514 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.469498    6116 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-envoy\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:17.469596 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.469516    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txcmd\" (UniqueName: \"kubernetes.io/projected/bee69a37-fadb-462f-9514-e8cb641fe4b1-kube-api-access-txcmd\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:17.469596 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.469526    6116 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bee69a37-fadb-462f-9514-e8cb641fe4b1-istiod-ca-cert\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:17.469596 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.469536    6116 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-podinfo\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:17.469596 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.469546    6116 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bee69a37-fadb-462f-9514-e8cb641fe4b1-credential-socket\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:17.469596 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.469554    6116 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bee69a37-fadb-462f-9514-e8cb641fe4b1-istio-token\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:17.469596 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.469543    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f051c9f2-e56c-4ced-b88d-b85747f3254f-plugins-config-volume" pod "f051c9f2-e56c-4ced-b88d-b85747f3254f" (UID: "f051c9f2-e56c-4ced-b88d-b85747f3254f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:00:17.471570 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.471539    6116 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f051c9f2-e56c-4ced-b88d-b85747f3254f-kube-api-access-9qdwj" pod "f051c9f2-e56c-4ced-b88d-b85747f3254f" (UID: "f051c9f2-e56c-4ced-b88d-b85747f3254f"). InnerVolumeSpecName "kube-api-access-9qdwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:00:17.570297 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.570263    6116 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f051c9f2-e56c-4ced-b88d-b85747f3254f-plugins-config-volume\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:17.570297 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.570294    6116 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qdwj\" (UniqueName: \"kubernetes.io/projected/f051c9f2-e56c-4ced-b88d-b85747f3254f-kube-api-access-9qdwj\") on node \"ip-10-0-129-138.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:00:17.889215 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.889182    6116 generic.go:356] "Generic (PLEG): container finished" podID="f051c9f2-e56c-4ced-b88d-b85747f3254f" containerID="d044c4c1dfc6471107413392659fa3c409bdf6f836e56b5c1c26568b7f0f6f35" exitCode=0
Nov 04 02:00:17.889393 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.889223    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt"
Nov 04 02:00:17.889393 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.889266    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt" event={"ID":"f051c9f2-e56c-4ced-b88d-b85747f3254f","Type":"ContainerDied","Data":"d044c4c1dfc6471107413392659fa3c409bdf6f836e56b5c1c26568b7f0f6f35"}
Nov 04 02:00:17.889393 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.889317    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt" event={"ID":"f051c9f2-e56c-4ced-b88d-b85747f3254f","Type":"ContainerDied","Data":"4416f89c48782e1d197c37e4ad0c5e24863964fb58bed0c0fa269c15729e276d"}
Nov 04 02:00:17.889393 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.889336    6116 scope.go:122] "RemoveContainer" containerID="d044c4c1dfc6471107413392659fa3c409bdf6f836e56b5c1c26568b7f0f6f35"
Nov 04 02:00:17.890645 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.890622    6116 generic.go:356] "Generic (PLEG): container finished" podID="aaa5bb34-fdba-497a-b58b-9ca549fa3d18" containerID="f7bce7de817c496175624931d6f5883e8d21d0290825947f97952edc8ab38da4" exitCode=0
Nov 04 02:00:17.890750 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.890670    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh"
Nov 04 02:00:17.890750 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.890701    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh" event={"ID":"aaa5bb34-fdba-497a-b58b-9ca549fa3d18","Type":"ContainerDied","Data":"f7bce7de817c496175624931d6f5883e8d21d0290825947f97952edc8ab38da4"}
Nov 04 02:00:17.890750 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.890737    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh" event={"ID":"aaa5bb34-fdba-497a-b58b-9ca549fa3d18","Type":"ContainerDied","Data":"2741cf04ff3a780a57116204463431946e13bed3f6b3f0f0bdb31d53aedcd31a"}
Nov 04 02:00:17.892158 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.892135    6116 generic.go:356] "Generic (PLEG): container finished" podID="bee69a37-fadb-462f-9514-e8cb641fe4b1" containerID="0f4885944b2769a4c5dccbbe90fc16993e57fff996b7934e4c8d5e87cb8bf6d3" exitCode=0
Nov 04 02:00:17.892265 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.892183    6116 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"
Nov 04 02:00:17.892265 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.892215    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w" event={"ID":"bee69a37-fadb-462f-9514-e8cb641fe4b1","Type":"ContainerDied","Data":"0f4885944b2769a4c5dccbbe90fc16993e57fff996b7934e4c8d5e87cb8bf6d3"}
Nov 04 02:00:17.892265 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.892253    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w" event={"ID":"bee69a37-fadb-462f-9514-e8cb641fe4b1","Type":"ContainerDied","Data":"67f8027182f6607c79f52c3fa385d3bc4345b65617b0f2e2329c99dd8f756127"}
Nov 04 02:00:17.900340 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.900323    6116 scope.go:122] "RemoveContainer" containerID="d044c4c1dfc6471107413392659fa3c409bdf6f836e56b5c1c26568b7f0f6f35"
Nov 04 02:00:17.900701 ip-10-0-129-138 kubenswrapper[6116]: E1104 02:00:17.900683    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d044c4c1dfc6471107413392659fa3c409bdf6f836e56b5c1c26568b7f0f6f35\": container with ID starting with d044c4c1dfc6471107413392659fa3c409bdf6f836e56b5c1c26568b7f0f6f35 not found: ID does not exist" containerID="d044c4c1dfc6471107413392659fa3c409bdf6f836e56b5c1c26568b7f0f6f35"
Nov 04 02:00:17.900781 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.900708    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d044c4c1dfc6471107413392659fa3c409bdf6f836e56b5c1c26568b7f0f6f35"} err="failed to get container status \"d044c4c1dfc6471107413392659fa3c409bdf6f836e56b5c1c26568b7f0f6f35\": rpc error: code = NotFound desc = could not find container \"d044c4c1dfc6471107413392659fa3c409bdf6f836e56b5c1c26568b7f0f6f35\": container with ID starting with d044c4c1dfc6471107413392659fa3c409bdf6f836e56b5c1c26568b7f0f6f35 not found: ID does not exist"
Nov 04 02:00:17.900781 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.900724    6116 scope.go:122] "RemoveContainer" containerID="f7bce7de817c496175624931d6f5883e8d21d0290825947f97952edc8ab38da4"
Nov 04 02:00:17.909248 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.909228    6116 scope.go:122] "RemoveContainer" containerID="f7bce7de817c496175624931d6f5883e8d21d0290825947f97952edc8ab38da4"
Nov 04 02:00:17.909524 ip-10-0-129-138 kubenswrapper[6116]: E1104 02:00:17.909502    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7bce7de817c496175624931d6f5883e8d21d0290825947f97952edc8ab38da4\": container with ID starting with f7bce7de817c496175624931d6f5883e8d21d0290825947f97952edc8ab38da4 not found: ID does not exist" containerID="f7bce7de817c496175624931d6f5883e8d21d0290825947f97952edc8ab38da4"
Nov 04 02:00:17.909574 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.909531    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7bce7de817c496175624931d6f5883e8d21d0290825947f97952edc8ab38da4"} err="failed to get container status \"f7bce7de817c496175624931d6f5883e8d21d0290825947f97952edc8ab38da4\": rpc error: code = NotFound desc = could not find container \"f7bce7de817c496175624931d6f5883e8d21d0290825947f97952edc8ab38da4\": container with ID starting with f7bce7de817c496175624931d6f5883e8d21d0290825947f97952edc8ab38da4 not found: ID does not exist"
Nov 04 02:00:17.909574 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.909547    6116 scope.go:122] "RemoveContainer" containerID="0f4885944b2769a4c5dccbbe90fc16993e57fff996b7934e4c8d5e87cb8bf6d3"
Nov 04 02:00:17.917541 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.917523    6116 scope.go:122] "RemoveContainer" containerID="0f4885944b2769a4c5dccbbe90fc16993e57fff996b7934e4c8d5e87cb8bf6d3"
Nov 04 02:00:17.917794 ip-10-0-129-138 kubenswrapper[6116]: E1104 02:00:17.917777    6116 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f4885944b2769a4c5dccbbe90fc16993e57fff996b7934e4c8d5e87cb8bf6d3\": container with ID starting with 0f4885944b2769a4c5dccbbe90fc16993e57fff996b7934e4c8d5e87cb8bf6d3 not found: ID does not exist" containerID="0f4885944b2769a4c5dccbbe90fc16993e57fff996b7934e4c8d5e87cb8bf6d3"
Nov 04 02:00:17.917858 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:17.917800    6116 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f4885944b2769a4c5dccbbe90fc16993e57fff996b7934e4c8d5e87cb8bf6d3"} err="failed to get container status \"0f4885944b2769a4c5dccbbe90fc16993e57fff996b7934e4c8d5e87cb8bf6d3\": rpc error: code = NotFound desc = could not find container \"0f4885944b2769a4c5dccbbe90fc16993e57fff996b7934e4c8d5e87cb8bf6d3\": container with ID starting with 0f4885944b2769a4c5dccbbe90fc16993e57fff996b7934e4c8d5e87cb8bf6d3 not found: ID does not exist"
Nov 04 02:00:18.054761 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:18.054731    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"]
Nov 04 02:00:18.104404 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:18.104374    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w"]
Nov 04 02:00:18.168367 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:18.168329    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt"]
Nov 04 02:00:18.172663 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:18.172636    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt"]
Nov 04 02:00:18.283501 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:18.283443    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh"]
Nov 04 02:00:18.287330 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:18.287305    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh"]
Nov 04 02:00:19.027602 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:19.027573    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aaa5bb34-fdba-497a-b58b-9ca549fa3d18" path="/var/lib/kubelet/pods/aaa5bb34-fdba-497a-b58b-9ca549fa3d18/volumes"
Nov 04 02:00:19.028005 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:19.027992    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bee69a37-fadb-462f-9514-e8cb641fe4b1" path="/var/lib/kubelet/pods/bee69a37-fadb-462f-9514-e8cb641fe4b1/volumes"
Nov 04 02:00:19.028298 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:19.028286    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f051c9f2-e56c-4ced-b88d-b85747f3254f" path="/var/lib/kubelet/pods/f051c9f2-e56c-4ced-b88d-b85747f3254f/volumes"
Nov 04 02:00:22.064648 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:22.064613    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-mzk5s"]
Nov 04 02:00:38.816508 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:38.816475    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6d4cf6d5f-9wgr6_e62dea92-10fa-4569-9477-2010e4897b58/manager/0.log"
Nov 04 02:00:40.340808 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:40.340780    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-lgdkj_74baa758-c88e-4814-926d-a59456774d5f/server/0.log"
Nov 04 02:00:40.582759 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:40.582727    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-l5vpc_61e56398-a014-42ad-89e9-dbd7fa34ec59/manager/0.log"
Nov 04 02:00:40.849264 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:40.849235    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-4qznm_d6ffa44d-084d-4df2-962c-d1f8cc30507e/proxy/0.log"
Nov 04 02:00:41.991854 ip-10-0-129-138 kubenswrapper[6116]: E1104 02:00:41.991817    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 02:00:42.770715 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:42.770684    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-287cf977-rk7mz_654a6571-b6fd-40f0-a369-87a487f1b508/revoke-keys/0.log"
Nov 04 02:00:42.999326 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:42.999296    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-39dbe6fd-8xr7w_cedbd40c-2a91-4928-ab9c-a2f647d3896d/revoke-keys/0.log"
Nov 04 02:00:43.231753 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:43.231723    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-57ae934e-prsjc_1f4bd2f5-7027-4f62-8123-b222884e9bbe/revoke-keys/0.log"
Nov 04 02:00:43.464428 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:43.464400    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-65edf5d0-f7bhm_6ddcfdc7-9d1d-4698-b43d-6a673d31497a/revoke-keys/0.log"
Nov 04 02:00:43.700992 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:43.700964    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-69b62d78-f4xgf_863ad259-3b19-4a29-bacf-5a6c8bc2222c/revoke-keys/0.log"
Nov 04 02:00:43.935553 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:43.935528    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-7f4748e0-6rpdw_9c6d88e0-1752-407e-8718-c8a64bce770a/revoke-keys/0.log"
Nov 04 02:00:44.157761 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:44.157738    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-8019849f-sgjms_929d4558-6de3-4bf9-894a-169498b5ed7b/revoke-keys/0.log"
Nov 04 02:00:44.387347 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:44.387313    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-acfc6aca-5bwkn_044a1757-03c3-4964-a3ab-71c685253791/revoke-keys/0.log"
Nov 04 02:00:48.604100 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:48.604070    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-mzk5s_66494911-a785-4301-9401-c5ba28818edb/limitador/0.log"
Nov 04 02:00:48.836217 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:48.836116    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-jt25s_451d239a-b06e-4ca3-af9b-7affd25cb5ff/manager/0.log"
Nov 04 02:00:49.902400 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:49.902367    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-58c98c4485dftg7_66431dd6-1363-4794-972e-9d274b31dfc1/istio-proxy/0.log"
Nov 04 02:00:50.734772 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:50.734745    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84fbf4d7ff-zlfjk_599691e1-af11-4605-b63b-3dd597be5801/kube-auth-proxy/0.log"
Nov 04 02:00:50.966707 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:50.966679    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-5x8z8_97871e88-cdc1-43ca-aca2-e3bbd128b086/istio-proxy/0.log"
Nov 04 02:00:51.719282 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:51.719255    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-m59rh_2376946a-54d9-4819-b016-d91a718df43a/istio-proxy/0.log"
Nov 04 02:00:52.479015 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:52.478989    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-qnxqv_41006a52-5c19-43e2-ba93-a2b1ca062162/istio-proxy/0.log"
Nov 04 02:00:52.986565 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:52.986530    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-s8k5g_437e07b5-3594-4bd4-b035-49047e446a61/istio-proxy/0.log"
Nov 04 02:00:53.451010 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:53.450983    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-75b555675f-xx482_3775ba44-0207-42df-830d-ac56e49f6a49/istio-proxy/0.log"
Nov 04 02:00:53.691823 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:53.691782    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-8ndrn_e2fa444a-b78e-4836-bc3e-61cb35f69987/payload-pre-processing/0.log"
Nov 04 02:00:53.910209 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:53.910185    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-tv42d_91242f71-8f7b-4743-a636-1a45d4fbcf89/payload-processing/0.log"
Nov 04 02:00:55.516814 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:55.516784    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6_f38bdf5d-3131-4d09-bac3-fc11735adf16/main/0.log"
Nov 04 02:00:56.040752 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:56.040721    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-xgzhk_255db273-54b3-4fa6-bdf1-e84cd275b05d/main/0.log"
Nov 04 02:00:56.281351 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:56.281320    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-z22gn_62f552a2-733d-419e-b9d2-6fa8a2246d6e/main/0.log"
Nov 04 02:00:57.295656 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:00:57.295618    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk_b0ba288d-0e75-4e9e-888a-10c1847a4ef2/main/0.log"
Nov 04 02:01:11.989397 ip-10-0-129-138 kubenswrapper[6116]: E1104 02:01:11.989355    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 02:01:15.495806 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:15.495771    6116 scope.go:122] "RemoveContainer" containerID="9063bef9f2120c3d8c126f6571483481a1cdbe7edbdf2dad33cb4f04e1a03521"
Nov 04 02:01:15.504790 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:15.504766    6116 scope.go:122] "RemoveContainer" containerID="eeffd11c7335008162a58f11eb7e21122943d788d5216bd753797b0cf4e5a429"
Nov 04 02:01:15.512981 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:15.512962    6116 scope.go:122] "RemoveContainer" containerID="abc7e831c38620be19f82b6c205ab01f542c7e0391fed7fb01f72411cc95c7cb"
Nov 04 02:01:17.603868 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:17.603837    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hjqq9_134fc8ea-90c5-4b4c-86a3-7b83d950f3e5/global-pull-secret-syncer/0.log"
Nov 04 02:01:20.793038 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:20.793005    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6d4cf6d5f-9wgr6_e62dea92-10fa-4569-9477-2010e4897b58/manager/0.log"
Nov 04 02:01:21.917249 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:21.917206    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-lgdkj_74baa758-c88e-4814-926d-a59456774d5f/server/0.log"
Nov 04 02:01:21.924496 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:21.924465    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-kwbdd_480e250d-44a3-4d5c-987d-22737107544e/konnectivity-agent/0.log"
Nov 04 02:01:22.374626 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:22.374547    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-138.us-west-1.compute.internal_5d9a5980aefdf4e226d40970221f0eb6/haproxy/0.log"
Nov 04 02:01:22.519947 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:22.519912    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-l5vpc_61e56398-a014-42ad-89e9-dbd7fa34ec59/manager/0.log"
Nov 04 02:01:22.630478 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:22.630381    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-4qznm_d6ffa44d-084d-4df2-962c-d1f8cc30507e/proxy/0.log"
Nov 04 02:01:25.569047 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:25.569016    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6d4cf6d5f-9wgr6_e62dea92-10fa-4569-9477-2010e4897b58/manager/0.log"
Nov 04 02:01:26.743930 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:26.743901    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-lgdkj_74baa758-c88e-4814-926d-a59456774d5f/server/0.log"
Nov 04 02:01:27.317662 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.317626    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-l5vpc_61e56398-a014-42ad-89e9-dbd7fa34ec59/manager/0.log"
Nov 04 02:01:27.445396 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.445342    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-4qznm_d6ffa44d-084d-4df2-962c-d1f8cc30507e/proxy/0.log"
Nov 04 02:01:27.811950 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.811910    6116 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"]
Nov 04 02:01:27.812426 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.812406    6116 cpu_manager.go:414] "containerMap: removing container" podUID="aaa5bb34-fdba-497a-b58b-9ca549fa3d18" containerName="payload-processing"
Nov 04 02:01:27.812426 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.812421    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aaa5bb34-fdba-497a-b58b-9ca549fa3d18" containerName="payload-processing"
Nov 04 02:01:27.812548 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.812488    6116 cpu_manager.go:414] "containerMap: removing container" podUID="f051c9f2-e56c-4ced-b88d-b85747f3254f" containerName="payload-pre-processing"
Nov 04 02:01:27.812548 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.812497    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f051c9f2-e56c-4ced-b88d-b85747f3254f" containerName="payload-pre-processing"
Nov 04 02:01:27.812548 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.812514    6116 cpu_manager.go:414] "containerMap: removing container" podUID="bee69a37-fadb-462f-9514-e8cb641fe4b1" containerName="istio-proxy"
Nov 04 02:01:27.812548 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.812519    6116 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bee69a37-fadb-462f-9514-e8cb641fe4b1" containerName="istio-proxy"
Nov 04 02:01:27.812661 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.812599    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="bee69a37-fadb-462f-9514-e8cb641fe4b1" containerName="istio-proxy"
Nov 04 02:01:27.812661 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.812612    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="aaa5bb34-fdba-497a-b58b-9ca549fa3d18" containerName="payload-processing"
Nov 04 02:01:27.812661 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.812620    6116 memory_manager.go:365] "RemoveStaleState removing state" podUID="f051c9f2-e56c-4ced-b88d-b85747f3254f" containerName="payload-pre-processing"
Nov 04 02:01:27.817352 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.817332    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:27.831151 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.831127    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/963114e2-8511-4022-8774-3b1c7a38e119-lib-modules\") pod \"perf-node-gather-daemonset-f9vfq\" (UID: \"963114e2-8511-4022-8774-3b1c7a38e119\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:27.831281 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.831177    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/963114e2-8511-4022-8774-3b1c7a38e119-sys\") pod \"perf-node-gather-daemonset-f9vfq\" (UID: \"963114e2-8511-4022-8774-3b1c7a38e119\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:27.831281 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.831194    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2lz6\" (UniqueName: \"kubernetes.io/projected/963114e2-8511-4022-8774-3b1c7a38e119-kube-api-access-x2lz6\") pod \"perf-node-gather-daemonset-f9vfq\" (UID: \"963114e2-8511-4022-8774-3b1c7a38e119\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:27.831369 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.831332    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/963114e2-8511-4022-8774-3b1c7a38e119-proc\") pod \"perf-node-gather-daemonset-f9vfq\" (UID: \"963114e2-8511-4022-8774-3b1c7a38e119\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:27.831369 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.831359    6116 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/963114e2-8511-4022-8774-3b1c7a38e119-podres\") pod \"perf-node-gather-daemonset-f9vfq\" (UID: \"963114e2-8511-4022-8774-3b1c7a38e119\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:27.920145 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.920110    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t4ldv\"/\"kube-root-ca.crt\""
Nov 04 02:01:27.920145 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.920110    6116 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t4ldv\"/\"openshift-service-ca.crt\""
Nov 04 02:01:27.920357 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.920179    6116 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-t4ldv\"/\"default-dockercfg-4lxxc\""
Nov 04 02:01:27.922600 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.922577    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"]
Nov 04 02:01:27.932129 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.932107    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/963114e2-8511-4022-8774-3b1c7a38e119-lib-modules\") pod \"perf-node-gather-daemonset-f9vfq\" (UID: \"963114e2-8511-4022-8774-3b1c7a38e119\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:27.932217 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.932162    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/963114e2-8511-4022-8774-3b1c7a38e119-sys\") pod \"perf-node-gather-daemonset-f9vfq\" (UID: \"963114e2-8511-4022-8774-3b1c7a38e119\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:27.932217 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.932189    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2lz6\" (UniqueName: \"kubernetes.io/projected/963114e2-8511-4022-8774-3b1c7a38e119-kube-api-access-x2lz6\") pod \"perf-node-gather-daemonset-f9vfq\" (UID: \"963114e2-8511-4022-8774-3b1c7a38e119\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:27.932314 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.932244    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/963114e2-8511-4022-8774-3b1c7a38e119-sys\") pod \"perf-node-gather-daemonset-f9vfq\" (UID: \"963114e2-8511-4022-8774-3b1c7a38e119\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:27.932314 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.932256    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/963114e2-8511-4022-8774-3b1c7a38e119-proc\") pod \"perf-node-gather-daemonset-f9vfq\" (UID: \"963114e2-8511-4022-8774-3b1c7a38e119\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:27.932314 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.932248    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/963114e2-8511-4022-8774-3b1c7a38e119-lib-modules\") pod \"perf-node-gather-daemonset-f9vfq\" (UID: \"963114e2-8511-4022-8774-3b1c7a38e119\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:27.932466 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.932310    6116 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/963114e2-8511-4022-8774-3b1c7a38e119-podres\") pod \"perf-node-gather-daemonset-f9vfq\" (UID: \"963114e2-8511-4022-8774-3b1c7a38e119\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:27.932466 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.932329    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/963114e2-8511-4022-8774-3b1c7a38e119-proc\") pod \"perf-node-gather-daemonset-f9vfq\" (UID: \"963114e2-8511-4022-8774-3b1c7a38e119\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:27.932466 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:27.932393    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/963114e2-8511-4022-8774-3b1c7a38e119-podres\") pod \"perf-node-gather-daemonset-f9vfq\" (UID: \"963114e2-8511-4022-8774-3b1c7a38e119\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:28.001971 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:28.001936    6116 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2lz6\" (UniqueName: \"kubernetes.io/projected/963114e2-8511-4022-8774-3b1c7a38e119-kube-api-access-x2lz6\") pod \"perf-node-gather-daemonset-f9vfq\" (UID: \"963114e2-8511-4022-8774-3b1c7a38e119\") " pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:28.129980 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:28.129898    6116 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:28.509947 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:28.509916    6116 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"]
Nov 04 02:01:28.511722 ip-10-0-129-138 kubenswrapper[6116]: W1104 02:01:28.511697    6116 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod963114e2_8511_4022_8774_3b1c7a38e119.slice/crio-a3441a9abd9c45da0c10413839fd9feb93af79df4da2fac2bd7f6fbaa963b7a8 WatchSource:0}: Error finding container a3441a9abd9c45da0c10413839fd9feb93af79df4da2fac2bd7f6fbaa963b7a8: Status 404 returned error can't find the container with id a3441a9abd9c45da0c10413839fd9feb93af79df4da2fac2bd7f6fbaa963b7a8
Nov 04 02:01:29.257085 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:29.257055    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-mzk5s_66494911-a785-4301-9401-c5ba28818edb/limitador/0.log"
Nov 04 02:01:29.301429 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:29.301390    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq" event={"ID":"963114e2-8511-4022-8774-3b1c7a38e119","Type":"ContainerStarted","Data":"7450289b3efd040993134945971a4b49cbc5b68a22d3b69282fdb469d40b1016"}
Nov 04 02:01:29.301625 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:29.301464    6116 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq" event={"ID":"963114e2-8511-4022-8774-3b1c7a38e119","Type":"ContainerStarted","Data":"a3441a9abd9c45da0c10413839fd9feb93af79df4da2fac2bd7f6fbaa963b7a8"}
Nov 04 02:01:29.301625 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:29.301492    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:29.418485 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:29.418401    6116 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq" podStartSLOduration=2.418385696 podStartE2EDuration="2.418385696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:01:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:29.416673425 +0000 UTC m=+1574.537950955" watchObservedRunningTime="2026-11-04 02:01:29.418385696 +0000 UTC m=+1574.539663270"
Nov 04 02:01:29.690229 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:29.690188    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-jt25s_451d239a-b06e-4ca3-af9b-7affd25cb5ff/manager/0.log"
Nov 04 02:01:30.419726 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:30.419694    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6d4cf6d5f-9wgr6_e62dea92-10fa-4569-9477-2010e4897b58/manager/0.log"
Nov 04 02:01:31.509850 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:31.509821    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-lgdkj_74baa758-c88e-4814-926d-a59456774d5f/server/0.log"
Nov 04 02:01:32.160245 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:32.160205    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-l5vpc_61e56398-a014-42ad-89e9-dbd7fa34ec59/manager/0.log"
Nov 04 02:01:32.289802 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:32.289762    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-4qznm_d6ffa44d-084d-4df2-962c-d1f8cc30507e/proxy/0.log"
Nov 04 02:01:35.316616 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:35.316586    6116 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-t4ldv/perf-node-gather-daemonset-f9vfq"
Nov 04 02:01:35.611102 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:35.611024    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-92fch_469986a0-2b14-47a4-8f4d-3a1456447cdb/kube-state-metrics/0.log"
Nov 04 02:01:35.615838 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:35.615815    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-58c98c4485dftg7_66431dd6-1363-4794-972e-9d274b31dfc1/istio-proxy/0.log"
Nov 04 02:01:36.021476 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:36.021420    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-92fch_469986a0-2b14-47a4-8f4d-3a1456447cdb/kube-rbac-proxy-main/0.log"
Nov 04 02:01:36.440172 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:36.440141    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-92fch_469986a0-2b14-47a4-8f4d-3a1456447cdb/kube-rbac-proxy-self/0.log"
Nov 04 02:01:37.138868 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:37.138833    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84fbf4d7ff-zlfjk_599691e1-af11-4605-b63b-3dd597be5801/kube-auth-proxy/0.log"
Nov 04 02:01:37.280521 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:37.280493    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-w4mgl_0b80cbf4-91a6-4716-ace2-9fd6d52ac7bc/monitoring-plugin/0.log"
Nov 04 02:01:37.356772 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:37.356735    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-5x8z8_97871e88-cdc1-43ca-aca2-e3bbd128b086/istio-proxy/0.log"
Nov 04 02:01:37.705211 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:37.705181    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-27c6h_4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f/node-exporter/0.log"
Nov 04 02:01:38.024786 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:38.024694    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-m59rh_2376946a-54d9-4819-b016-d91a718df43a/istio-proxy/0.log"
Nov 04 02:01:38.123687 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:38.123661    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-27c6h_4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f/kube-rbac-proxy/0.log"
Nov 04 02:01:38.544873 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:38.544848    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-27c6h_4ecc4bea-722e-4b62-a5f9-76cc8e5d9a4f/init-textfile/0.log"
Nov 04 02:01:38.670184 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:38.670156    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-qnxqv_41006a52-5c19-43e2-ba93-a2b1ca062162/istio-proxy/0.log"
Nov 04 02:01:39.103810 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:39.103780    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-s8k5g_437e07b5-3594-4bd4-b035-49047e446a61/istio-proxy/0.log"
Nov 04 02:01:39.431249 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:39.431220    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-75b555675f-xx482_3775ba44-0207-42df-830d-ac56e49f6a49/istio-proxy/0.log"
Nov 04 02:01:39.644611 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:39.644572    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-8ndrn_e2fa444a-b78e-4836-bc3e-61cb35f69987/payload-pre-processing/0.log"
Nov 04 02:01:39.859748 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:39.859669    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-tv42d_91242f71-8f7b-4743-a636-1a45d4fbcf89/payload-processing/0.log"
Nov 04 02:01:41.994941 ip-10-0-129-138 kubenswrapper[6116]: E1104 02:01:41.994900    6116 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8zjpf" containerName="extractor"
Nov 04 02:01:42.762038 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:42.762006    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/prometheus/0.log"
Nov 04 02:01:42.762945 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:42.762921    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/prometheus/1.log"
Nov 04 02:01:43.069255 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:43.069162    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn"]
Nov 04 02:01:43.093393 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:43.093363    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/config-reloader/0.log"
Nov 04 02:01:43.118930 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:43.118892    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-acfc6aca-5bwkn"]
Nov 04 02:01:43.514811 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:43.514758    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/thanos-sidecar/0.log"
Nov 04 02:01:43.944046 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:43.944018    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/kube-rbac-proxy-web/0.log"
Nov 04 02:01:44.368961 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:44.368881    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/kube-rbac-proxy/0.log"
Nov 04 02:01:44.812254 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:44.812227    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/kube-rbac-proxy-thanos/0.log"
Nov 04 02:01:45.034834 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:45.034794    6116 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="044a1757-03c3-4964-a3ab-71c685253791" path="/var/lib/kubelet/pods/044a1757-03c3-4964-a3ab-71c685253791/volumes"
Nov 04 02:01:45.140999 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:45.140914    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-mzk5s_66494911-a785-4301-9401-c5ba28818edb/limitador/0.log"
Nov 04 02:01:45.241776 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:45.241740    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d586274f-5bbb-468e-9eba-3d6809e4c78a/init-config-reloader/0.log"
Nov 04 02:01:45.371244 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:45.371209    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-jt25s_451d239a-b06e-4ca3-af9b-7affd25cb5ff/manager/0.log"
Nov 04 02:01:45.664593 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:45.664555    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-mgxhd_eb15eb62-434d-4cdd-a2dd-d1e069bb6abb/prometheus-operator/0.log"
Nov 04 02:01:46.078602 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:46.078525    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-mgxhd_eb15eb62-434d-4cdd-a2dd-d1e069bb6abb/kube-rbac-proxy/0.log"
Nov 04 02:01:50.743932 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:50.743897    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-58c98c4485dftg7_66431dd6-1363-4794-972e-9d274b31dfc1/istio-proxy/0.log"
Nov 04 02:01:51.046531 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:51.046414    6116 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz"]
Nov 04 02:01:51.095657 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:51.095617    6116 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-287cf977-rk7mz"]
Nov 04 02:01:52.084948 ip-10-0-129-138 kubenswrapper[6116]: I1104 02:01:52.084914    6116 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84fbf4d7ff-zlfjk_599691e1-af11-4605-b63b-3dd597be5801/kube-auth-proxy/0.log"